Laravel Scout & Elasticsearch: Scout flush does not delete all records

If you've been using Laravel Scout for Elasticsearch, there's a chance you have faced this issue in development. This also probably applies to all other drivers - Meilisearch, Algolia.

Let's say you have a MySQL table of records that's synced to your search index. Every time you write or update a record, this calls the searchable and unsearchable method to keep it in sync, with SoftDeletes if you have that Trait included.

If you add a record manually to your index using the UI for Elasticsearch or any other driver, but not to the database - Scout::flush() will not remove the entry from the index.

It's important to understand why this happens. When you sync, Laravel Scout matches every record with the unique ID of the model (usually the default key - can be overriden with getScoutKey). Similarly, when you delete records, Scout fetches keys from your table and matches it with index to delete them, therefore it retains the record in your index which is not present in the table.