A number of weeks in the past, I ran my standard suite of programming tests towards the free model of the Perplexity.ai chatbot. At the end of the article, I provided to run checks towards the $20/mo Pro version if sufficient of you have been . I did get some requests, so that is what we’re doing right here.
Additionally: Want a free year of Perplexity Pro? Here’s how to claim it
Like most different professional variations, to make use of Perplexity Professional, it’s important to create an account. You’ll be able to register utilizing both Google or Apple auth strategies or a SAML sign-in. Alternatively, you’ll be able to create an account utilizing your electronic mail deal with, which is what I did.
Sadly, the location does not appear to offer you any approach to set a password or any type of multifactor authentication. You are despatched an electronic mail with a code, and that is it. I do not thoughts getting an electronic mail code, however I am actually disturbed by net apps relying solely on an electronic mail code with out, no less than, a password. However that is what Perplexity.AI is doing.
Additionally: 5 reasons why I prefer Perplexity over every other AI chatbot
The opposite attention-grabbing side of Perplexity Professional is its cornucopia of AI fashions. As you’ll be able to see within the picture beneath, you’ll be able to select between plenty of totally different fashions, based mostly on the type of work you may have. I selected Default to see what that did with the checks. After operating the checks, I requested Perplexity Professional what mannequin it used for them, and it advised me ChatGPT GPT-4.
And with that, let’s run some checks.
1. Writing a WordPress plugin
This problem is a reasonably easy programming process for anybody with a modicum of net programming expertise. It presents a consumer interface within the administration dashboard with two fields: one is a listing of names to be randomized, and the opposite is the output.
The one actual gotcha is that the record of names can have duplicates, and quite than eradicating the additional names, its directions are to verify the duplicate names are separated from one another.
Additionally: How I test an AI chatbot’s coding ability – and you can too
This was an actual, requested perform that my spouse wanted to make use of for her e-commerce web site. Each month, they do a wheel spin and a few individuals qualify for a number of entries.
Utilizing Perplexity Professional’s default mannequin, the AI succeeded in producing a workable consumer interface and purposeful code, offering each a PHP block and a JavaScript block to manage the textual content areas and the randomization logic.
Listed here are the combination outcomes of this and former checks:
- Perplexity Professional: Interface: good, performance: good
- Perplexity: Interface: good, performance: good
- Claude 3.5 Sonnet: Interface: good, performance: fail
- ChatGPT utilizing GPT-4o: Interface: good, performance: good
- Microsoft Copilot: Interface: ample, performance: fail
- Meta AI: Interface: ample, performance: fail
- Meta Code Llama: Full failure
- Google Gemini Superior: Interface: good, performance: fail
- ChatGPT utilizing GPT-4: Interface: good, performance: good
- ChatGPT utilizing GPT-3.5: Interface: good, performance: good
2. Rewriting a string perform
For every check, I open a brand new session with the AI. On this check, I am asking the AI to rewrite a block of code that had a bug. The code was designed to validate the enter of {dollars} and cents, which ought to comprise a sure variety of digits earlier than the decimal level, a doable decimal level, and two digits after the decimal level.
Additionally: Yikes! Microsoft Copilot failed every single one of my coding tests
Sadly, the code I shipped solely allowed integer numbers. After a few consumer experiences, I made a decision to feed the code to the AI for a rewrite. My code makes use of common expressions, that are a formulaic method of specifying a format. Common expressions themselves are enjoyable, however debugging them is just not.
Within the case of this check, Perplexity Professional did a very good job. The resultant validation code correctly flagged objects that didn’t match the format for {dollars} and cents, permitting as much as two digits after the decimal.
Listed here are the combination outcomes of this and former checks:
- Perplexity Professional: Succeeded
- Perplexity: Succeeded
- Claude 3.5 Sonnet: Failed
- ChatGPT utilizing GPT-4o: Succeeded
- Microsoft Copilot: Failed
- Meta AI: Failed
- Meta Code Llama: Succeeded
- Google Gemini Superior: Failed
- ChatGPT utilizing GPT-4: Succeeded
- ChatGPT utilizing GPT-3.5: Succeeded
3. Discovering an annoying bug
This check had me stumped for a number of hours. Earlier than it was a check, it was a bug within the code for an precise product. The issue was that no matter was going fallacious wasn’t associated to any apparent logic or language challenge.
Additionally: I asked ChatGPT to write a WordPress plugin I needed. It did it in less than 5 minutes
Being severely pissed off, I made a decision to feed ChatGPT the code in addition to the error dump and ask it for assist. Fortuitously, it discovered what I had carried out fallacious and gave me steering on what to repair.
The rationale I am together with this within the set of checks is as a result of the bug wasn’t in language or logic, it was in information of the WordPress framework. Whereas WordPress is widespread, framework information is usually thought-about the folklore of a programming atmosphere, one thing handed down from developer to developer, quite than one thing that may be rigorously realized by a information base.
Nevertheless, ChatGPT, in addition to Perplexity and now Perplexity Professional, did discover the issue. The error was on account of a parameter calling challenge buried within the framework itself. The apparent reply, which you may provide you with strictly by studying the error messages generated by the code, was really fallacious.
Additionally: Uber One subscribers get a free year of Perplexity Pro. Here’s how to claim it
To resolve it, the AI needed to present a deeper understanding of how all of the methods work collectively, one thing with Perplexity Professional did efficiently.
Listed here are the combination outcomes of this and former checks:
- Perplexity: Succeeded
- Perplexity Professional: Succeeded
- Claude 3.5 Sonnet: Succeeded
- ChatGPT utilizing GPT-4o: Succeeded
- Microsoft Copilot: Failed
- Meta AI: Succeeded
- Meta Code Llama: Failed
- Google Gemini Superior: Failed
- ChatGPT utilizing GPT-4: Succeeded
- ChatGPT utilizing GPT-3.5: Succeeded
4. Writing a script
Effectively, that is attention-grabbing. Perplexity Professional handed this check, however the free version of Perplexity failed once I examined it a few weeks in the past. So, yay!
However let’s dive into this a bit. The problem right here is that I ask the AI to write down a script that intersects three environments: the Chrome DOM (doc object mannequin), AppleScript (Apple’s native scripting language), and Keyboard Maestro (a really cool Mac automation software that is pretty obscure, however to me, mission-critical).
Many of the AIs failed as a result of they did not have any info on Keyboard Maestro of their information bases and, as such, did not give the mandatory code for the script to do what I wished.
Additionally: How to use ChatGPT to write code: What it can and can’t do for you
Solely Gemini Advanced and ChatGPT utilizing GPT-4 and GPT-4o handed this check till now. In answering the query, Perplexity Professional offered a Professional Search view. As you’ll be able to see, the Professional Search view did a seek for “Keyboard Maestro AppleScript Google Chrome tabs.” It additionally used the main Keyboard Maestro forum as a supply, which is the most effective supply for getting Keyboard Maestro coding assist.
The consequence was a hit.
Listed here are the combination outcomes of this and former checks:
- Perplexity Professional: Succeeded
- Perplexity: Failed
- Claude 3.5 Sonnet: Failed
- ChatGPT utilizing GPT-4o: Succeeded however with reservations
- Microsoft Copilot: Failed
- Meta AI: Failed
- Meta Code Llama: Failed
- Google Gemini Superior: Succeeded
- ChatGPT utilizing GPT-4: Succeeded
- ChatGPT utilizing GPT-3.5: Failed
Total outcomes
Listed here are the general outcomes of the 4 checks:
As you’ll be able to see, Perplexity Professional joins solely ChatGPT with GPT-4 and GPT-4o as having an ideal rating of 4 out of 4 succeeded. After operating my checks, I checked with Perplexity Professional’s AI and it knowledgeable me it used GPT-4 to research and reply to my checks.
On condition that GPT-4/4o is the one AI that nailed all 4 of my checks earlier than, this is sensible. Thus far, I have not discovered some other mannequin that may absolutely and accurately go all 4 programming checks.
Additionally: How to run dozens of AI models on your Mac or PC – no third-party cloud needed
Should you select Perplexity Professional, I can pretty confidently state that it ought to be capable to do a very good job of serving to you program.
Have you ever tried coding with Perplexity, Copilot, Meta AI, Gemini, or ChatGPT? What has your expertise been? Tell us within the feedback beneath.
You’ll be able to observe my day-to-day challenge updates on social media. Be sure you subscribe to my weekly update newsletter, and observe me on Twitter/X at @DavidGewirtz, on Fb at Facebook.com/DavidGewirtz, on Instagram at Instagram.com/DavidGewirtz, and on YouTube at YouTube.com/DavidGewirtzTV.
