Chapter 5: Sculpting Data In-Memory

While working with data, sometimes, for performance or convenience purposes, we must perform operations on the data that are not persistently stored to disk. For performance, an example is combining billions of rows of transactions with millions of rows of customer data. For convenience, an example might be a simple code mapping lookup or applying business logic. These fine-tuning adjustments are enabled by tools that work on in-memory caches and query technology. In Domo's case, this is Adrenaline, which is a queryable in-memory cache. These in-memory transforms are orders of magnitude faster than persistent transforms via ETL and when fast is what you are after, these tools are the right choice. Occasionally, the use of Adrenaline dataflows becomes critical in cases when the rows of data being processed reaches billions of rows, joins are required, and materialization is not practical. If you are familiar with creating non-materialized views...