Evaluating models
By now, we have gained a comprehensive understanding of Amazon Bedrock’s capabilities, exploring techniques such as prompt engineering, RAG, and model customization. We have also examined various architectural design patterns and analyzed the responses generated by different models. With the vast selection of FMs available within Amazon Bedrock, identifying the most suitable option for your specific use case and business requirements can be challenging. To address this, we will now focus on the topic of model evaluation and how to compare the outputs of different models to choose the one that best meets the needs of your application and business. This is a critical initial phase in implementing any generative AI solution.
Figure 11.1 – The generative AI life cycle
As shown in Figure 11.1, after defining the specific business use case that you’re aiming to solve with generative AI, the choice stage involves both selecting...
