Using models for various different purposes daily has been a satisfying endeavor for me because they can be used as tools to help make your vision for something come to life. Models are powerful generators that can produce code, writing, images and more based on a user’s description of what they want. But models “fill in the gaps” on behalf of the user to resolve ambiguity in the user’s prompt.I attempted to reproduce the results for one task from the VLMs are Blind paper. Specifically, Task 1: Counting line intersections. I ran 150 examples of lines generated by the code from the project with line thickness 4.
I started with the prompt:
How many times do the blue and red lines intersect?
using the model claude-3.5-sonnet with temperature 0. The paper reported 73.00% correctness for claude-3.5-sonnet with thickness 4.I’m trying something a bit new, writing some of my thoughts about how the future might look based on patterns I’ve been observing lately.
From where I’m sitting, it seems language models are positioned to become an indispensable tool for the software engineer. While there continues to be advancement in model-driven agents which can autonomously accomplish software-creation tasks of increasing complexity, it’s not clear if or how long it will take these to completely replace the job function responsible for writing, testing, deploying and maintaining software in a production environment.I spent some time working with Claude Artifacts for the first time. I started with this prompt
I want to see what you can do. Can you please create a 2d rendering of fluid moving around obstacles of different shapes?
In effort to not spend this whole post quoting prompts I need to figure out a better way to share conversations from all the different models I interact with, including multi-modal models.Temporal provides helpful primitives called Workflows and Activities for orchestrating processes. A common pattern I’ve found useful is the ability to run multiple “child workflows” in parallel from a single “parent” workflow.
Let’s say we have the following activity and workflow (imports omitted for brevity)
Activity code
@dataclass class MyGoodActivityArgs: arg1: str arg2: str @dataclass class MyGoodActivityResult: arg1: str arg2: str random_val: float @activity.defn async def my_good_activity(args: MyGoodActivityArgs) -> MyGoodActivityResult: activity.I spent some time experimenting with multi-modal model (also called vision models on the ollama site) to see how they perform. You try these out with the CLI ollama run <model> but I opted to use the ollama Python client.
I didn’t find explicit documentation in the README on how to pass images to the model but the type hints in the code made it pretty easy to figure out and there are several examples around Github.One of my favorite things to do with language models is to use them to write code. I’ve been wanting to build a variation on tic-tac-toe involving a bit of game theory. I called it “Tactic”. I wasn’t even really sure if the game would be any more interesting than tic-tac-toe itself, which reliably ends in draws for any players who understand the basics of the game. Rather than explain too much, I’ll show the prompt I wrote for claude-3.Model-based aggregators I watched Simon’s Language models on the command-line presentation. I am a big fan of his Unix-approach to LLMs. This also inspired me to play around more with smaller models to continue developing an intuition for how these things work.
I was quite interested in his script which he used to summarize comments on an orange site post at 26:35 in the video. This script got me thinking about the future of information consumption more deeply.I completed Barbara Oakley’s “Learning How to Learn” course on Coursera. The target audience seems to be students, but I found there were helpful takeaways for me as well, as someone who is a decade out of my last university classroom.
The course introduces a mental model (no pun intended) for how the brain works by contrasting two modes: focus mode and diffuse mode. Being in one of these modes prevents you from being in the other.I’ve been using Pocket for a long time to keep track of things on the web that I want to read later. I save articles on my mobile or from my browser, then revisit them, usually on my desktop. Some articles I get to quickly. Others remain in the stack for a long time and can become stale. Periodically, I scan through everything I’ve saved and do a bit of house cleaning.