Now that version 2.1 of the SketchUp OpenAI Explorer extension is available, it is easier to test the AI’s capabilities when using the most recent GPT-4 model. In this post, I will be comparing side-by-side the
gpt-3.5-turbo and the
gpt-4 models with a few common examples. If you want to give this a try, too, make sure you have access to the gpt-4 model and then simply replace the model name in my SketchUp extension’s settings dialog.
The extension page on my other website features a longer list of successful prompts than what I am able to cover here. Feel free to give those a try, too.
Example1: Draw a Box
For this example, I asked the AI to draw a 2′ (i.e. “two foot”) box. I wanted to test not only whether the box would get drawn correctly, but also if the foot tick mark would be interpreted correctly as the foot unit. The images below show what I got with that request:
While both runs created boxes (cubes, actually), the gpt-4 model interpreted the request better than the gpt-3.5 model. As you can see, the gpt-4 model even reversed the created face so that the box would be extruded upwards. It did all of this, however, at a significantly increased time (16.96 seconds vs. 6.19 seconds).
Example 2: Select Something
In this example I asked to select all component instances in a model (and not select anything else: faces, lines, etc.). I intentionally did not ask for “instances” and wanted to see whether the AI can correctly interpret my request as “instances” and not “definitions” (which cannot be selected, of course). This process is usually more reliable if you ask to clear the selection first, which is why I included “select none” in my request.
While gpt-4 again took longer to execute, the generated code is also again a bit cleaner (because of its use of the
.grep method). The result was the same for both runs, however: All of the component copies were selected.
Example 3: Colorize Objects
This request not only asked to create new SketchUp materials and assign them to entities in the model, it also required the AI to interpret a theme (i.e. “fall” in my case) and create a set of colors based on that.
Once again gpt-4 took much longer to execute but the code is a bit cleaner because it used the
.sample method on the color array (instead of creating a random index). The result is similar for both, however.
gpt-4 model appears to generate slightly better SketchUp Ruby code, the
gpt-3.5-turbo model generally creates good, usable code that is actually quite accurate in many cases. It is also much faster in executing the requests, and – quite importantly – it is much cheaper to use (see table with current pricing below).
|Input: $0.0015 / 1K tokens|
Output: $0.002 / 1K tokens
|Input: $0.03 / 1K tokens|
Output: $0.06 / 1K tokens
I hope this comparison helps you when you work with the SketchUp OpenAI Explorer extension. Did you do any comparisons yourself? Let me know in the comments below.