Kafka
April 2, 2016
Chapter 1 Preface
Section 0: Data Structure and Algorithm
Storage Engine:
- B+-Tree
- LSMT
- SkipList
rotating_skiplist.pdf - R-tree, Kd Tree, Quad Tree
- inverted index
- Others (Fractal Tree)
Section 1: Data Storage
OLTP
- RDBMS (MySQL, PostgreSQL)
- NoSQL (Cassandra, MongoDB)
- NewSQL (TiDB)
- In-Mem Store(Redis, MemSQL)
OLAP
Section 2: Data Representation and Evolution
- Json/XML
- messagepack
- protobuff
- flatbuffer
- thrift
- avro
- parquet-mr
- gRPC
Section 3: Data Transfer
- ZeroMQ
- REST
- gRPC
- Kafka
- RocketMQ
- RabbitMQ
Section 4: Data Processing/Computing
- Batching(hadoop MR)
- MicroBatching(Spark)
- Streaming(Storm, Heron)
- Others(Kafka Stream, Redis Stream)
Section 5: Machine Learning and AI
- Recommendation
- Clustering
Section 6: Product Design Topics
https://puncsky.com/hacking-the-software-engineer-interview/?isAdmin=true
Uber
https://eng.uber.com/
https://eng.uber.com/uber-big-data-platform/
https://eng.uber.com/mysql-migration/Facebook
https://eng.uber.com/sylvain-francois-uber-freight/
- Messenger System(slack, FB, whatsapp, linkedin messenger)
- Twitter
- Google Calendar
- Hangman
- TinyURL
- TypeAhead
- Excel
- Youtube
- TopK shared Pages
- Music Player(MFU,Shuffle)
- Pinterest or Instagram
- Facebook Photo storage
https://www.usenix.org/legacy/event/osdi10/tech/full_papers/Beaver.pdf