Summary

In this chapter, we discussed in-depth considerations for quality and temporal data concepts. All of the concepts we discussed have trade-offs, and it’s always worth taking a step back to confirm your requirements before making a design decision. Lat/long data quality, transmission methods, streaming versus batch, and big data file formats are core architecture decisions. Jeff Bezos has a famous theory about one-way doors and two-way doors: invest time in one-way door decisions, or essentially, design decisions that would be very disruptive to go back and change. Two-way doors are design decisions that are easy to change later. Many of the concepts in this chapter are one-way doors. If you collected all of your sensor data at 1-meter accuracy and you later realize you need 0.1-meter accuracy, it is probably not trivial to redeploy all your sensors to get more accurate readings.

Lastly, two more takeaways. Always keep in mind the 80/20 rule for analyzing and normalizing...