Building the future with LLMOps
Given the rise in interest in LLMs recently, there has been no shortage of people expressing the desire to integrate these models into all sorts of software systems. For us as ML engineers, this should immediately trigger us to ask the question, “What will that mean operationally?” As discussed throughout this book, the marrying together of operations and development of ML systems is termed MLOps. Working with LLMs is likely to lead to its own interesting challenges, however, and so a new term, LLMOps, has arisen to give this sub-field of MLOps some good marketing.
Is this really any different? I don’t think it is that different, but should be viewed as a sub-field of MLOps with its own additional challenges. Some of the main challenges that I see in this area are:
- Larger infrastructure, even for fine-tuning: As discussed previously, these models are far too large for typical organizations or teams to consider training...
