I read a post a couple of months ago (Sadly, I can’t find the link), about basic tooling for your daily work. You should use one product of every type of thing. This could be learning and using for example RabbitMQ for pub/sub, queues and such. If you ever need a go-to database it could be MariaDB etc. Monitoring could be Sensu etc. I have decided to do just that. I have started using RabbitMQ for processing messages that get sent to queues. Let’s say that I want to download a video from a video site. I will send a json-blob to a queue in RabbitMQ and some other process will look at that queue and process it the json data which contains a URL. The output will be delivered to a bucket in AWS S3. If the json-blob gets sent to a another queue, it will only process the video to extract the audio, and then it will send it to S3.
This is not a complete list, but it is a start:
- Queue, pub/sub: RabbitMQ
- Relational DB: MariaDB
- Object store: AWS S3
- Certificate Management: Smallstep
- NoSQL DB: MongoDB