With the merging of the below PR, DEV no longer requires Algolia to run!
Remove ALL Algolia Code
#7577
What type of PR is this? (check all applicable)
- [x] Refactor
- [x] Optimization
- [x] Documentation Update
Description
Remove all traces of Algolia from our codebase and add documentation for Elasticsearch. The Elasticsearch docs will likely need to be worked on as we go but I gave us what I thought was a good foundation.
Once this is out we can remove Algolia from Heroku!
Related Tickets & Documents
https://github.com/thepracticaldev/dev.to/projects/6#card-32955489 https://github.com/thepracticaldev/dev.to/projects/6#card-32912217
Added tests?
- [x] no, because they aren't needed
Added to documentation?
- [x] YES!
This is a huge step forward for our community because it means that those contributing no longer have to sign up for an Algolia account. Instead, contributors can simply download the open-source software from Elastic when they are setting up their development environments.
For more information on how DEV uses Elasticsearch, checkout our new docs page.
Happy coding!
Top comments (14)
It's really exciting to be on an open technology, generally vendor agnostic, and installable on our local machines.
I chose Algolia years ago for its remarkable performance, ease of use, and quality of client libraries. Those things are all true so I don't want anyone to see this as too black-and-white. Algolia is a fabulous tool. But as we need to evolve, this is. a big stepโof course, for developer experience, but also giving us more tools to work with in building a tremendously powerful tool together.
Thanks for this comment Ben. We made a similar choice for Flotiq, unlike most other headless CMSes we decided to incorporate ES into our API, so that developers can use its powers.
I would really love to learn a bit more about the motivation, what were the particular features you guys thought are most important and why.
๐๐๐๐๐๐ Awesome.
Great news!
Also wondering what was the reasoning for choosing Elasticsearch, I have recently learned about MeiliSearch and it looks very promising.
Elasticsearch is a tried and true technology that has been around for a while and has a solid Ruby library. It is also a database that many of the major platforms like AWS and GCP support and host which allows greater flexibility when users decide they want to spin up their own communities.
It is also something I have worked extensively on in the past and that added experience was what allowed us to make this transition very quickly bc I knew exactly what needed to happen and how to do it. We considered other tools such as the new Redis search but the fact that it was so young and unfamiliar made us decide to stick with what we knew.
Hooray! ๐ฅณ๐ฅณ๐ฅณ
Developer experience for the win!
We've had a difficult time trying to keep Elasticsearch working correctly on our EC2 instance. It takes a lot of work to get it configured right, especially if things break.
If you don't mind me asking, what are you using to host Elasticsearch, and why?
Right now we are using Elastic's hosted service. At my prior job our cluster was self-hosted on AWS. I could see us possibly going that route again so we have more control over our cluster.
If you are going to self-host I highly recommend going through an Elasticsearch training course beforehand. That will layout a lot of gotchas and can be super helpful to get you up and running.
That's good advice, thanks.
Great work, Molly :)
Achieving great stuff in no time. ๐
how much effort does it require to maintain in general? I have recently added algolia search to my web and while it works I do feel like I am overpaying for what it's doing :)
Could it just run on a single container in kubernetes with a mounted volume? I wouldn't care about the persistence much as I could rebuild the index every day.
Do you need to perform some daily/monthly cleanups on it?
It depends on how you use it but we dont do any maintenance on our index. The code handles adding and removing documents as needed to ensure it stays in sync with what is in our database.
Having a small ES cluster is easy to maintain. I like to say Elasticsearch is easy to learn and hard to master. The time you really have to know what your doing is if you are going to seriously scale it. Depending on document size a few hundred million docs is pretty easy to handle. Beyond that you want to know what you are doing.
If you do want to eventually scale checkout my scaling blog posts: dev.to/molly_struve/scaling-elasti...
Have you ever considered Apache Solr before adopting Elasticsearch? They are both built on the same search engine, and are capable of doing almost the same thing.
We definitely considered multiple technologies but in the end, given our use case and because of my expertise with Elasticsearch we decided it would be the best fit for us. We could ramp up quickly with it and then scale well into the future.