Estimating the potential cost of using metadata extractors
A key consideration when utilizing the various metadata extractors in LlamaIndex is the associated LLM compute costs. As mentioned earlier, most of these extractors rely on LLMs under the hood to analyze text and generate descriptive metadata.
Repeatedly calling LLMs to process large volumes of text can quickly add up in charges. For example, if you are extracting summaries and keywords from thousands of document nodes using SummaryExtractor and KeywordExtractor, those constant LLM invocations will carry a significant cost.
Follow these simple best practices to minimize your costs
Let’s talk about some common best practices for minimizing your LLM costs:
- Batch content into fewer LLM calls instead of individual calls per node. This amortizes the overhead because you consume fewer tokens compared to multiple separate calls. Using the Pydantic extractor is very useful for this purpose since it generates...
