Some unstructured thoughts on the types of tasks language models seem to be good (and bad) at completing:
A language model is an effective tool for solving problems when can describe the answer or output you want from it with language. A language model is a good candidate to replace manual processes performed by humans, where judgement or application of semantic rules is needed to get the right answer. Existing machine learning approaches are already good at classifying or predicting over a large number of features, specifically when one doesn’t know how things can or should be clustered or labelled just by looking at the data points. To give an example where a language model will likely not perform well: imagine you want to generate a prediction for the value of a house and the land it sits on, given a list of data points describing it:
- lat/long
- square footage
- build year
- number of bed and bathrooms
- lot size
- whether it has a pool
Traditional machine learning approaches presently can do a good job at this. A language model likely would not. While the token outputs from language models are based on numerical predictions, those tokens should not be interpreted as numerical predictions themselves. Token outputs are more of a qualitative assessment of the inputs rather than a quantitative one (ironic, given that it’s math all the way down).
Language models are a bad fit for this house-price-prediction task for the same reason that they struggle to do math.
Thanks to CSL for his post on getting full-content output in a Hugo RSS feed. This post helped me finally get around to fixing it on this site.