Today, I played around with Matt Rickard’s ReLLM library, another take on constraining LLM output, in this case, with regex. I tried to use it to steer a language model to generate structure (JSON) from unstructured input. This exercise is sort of like parsing or validating JSON with regex – it’s not a good idea. Complicated regular expressions to describe JSON are hard to write. I do love the demo that generates acronyms though. Matt also wrote parserLLM which provides to ability to use a context-free grammar to constrain the next predicted token from the language model. I prefer the context-free grammar approach at a high-level, but believe we need the language model constraints to be directly connected with the data structures we intend to use in code to effectively weave language models into our existing applications.