[IMPORTANT] Problématique recherche forum avec docker

[Cara62]

Bonjour à tous,

Depuis la migration du forum sur le nouveau serveur, @Edtech et moi avons décidé que la recherche elasticsearch du forum utilisera la version docker. 

Or, certains l'auront s'en doute remarqués la recherche ne fonctionne plus de temps en temps. 

Après investigation de mon côté, il s'avère qu'apparemment, régulièrement, elasticsearch invalide les data et du coup supprime l'indéxation. Ce qui vous comprendrez c'est un peu problématique. 😄

 

 Après des heures de recherches, je n'ai pas trouvé comment resoudre ce problème. Du coup si quelqu'un connait mieux la version docker d'elasticsearch pour nous aider. 🙂

 

Voici les derniers logs : 

 

  Révéler le contenu masqué

[2020-08-30T01:38:00,002][INFO ][o.e.x.m.MlDailyMaintenanceService] [esforum] triggering scheduled [ML] maintenance tasks
[2020-08-30T01:38:00,006][INFO ][o.e.x.m.a.TransportDeleteExpiredDataAction] [esforum] Deleting expired data
[2020-08-30T01:38:00,015][INFO ][o.e.x.m.a.TransportDeleteExpiredDataAction] [esforum] Completed deletion of expired ML data
[2020-08-30T01:38:00,016][INFO ][o.e.x.m.MlDailyMaintenanceService] [esforum] Successfully completed [ML] maintenance tasks
[2020-08-30T04:28:43,438][INFO ][o.e.c.m.MetaDataMappingService] [esforum] [content/5BnvJdR-RD22z2lSm1TmuQ] update_mapping [content]
[2020-08-30T04:28:43,520][INFO ][o.e.t.LoggingTaskListener] [esforum] 4687981 finished with response BulkByScrollResponse[took=3.4ms,timed_out=false,sliceId=null,updated=0,created=0,deleted=0,batches=0,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[2020-08-30T04:28:43,586][INFO ][o.e.t.LoggingTaskListener] [esforum] 4688002 finished with response BulkByScrollResponse[took=1.9ms,timed_out=false,sliceId=null,updated=0,created=0,deleted=0,batches=0,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[2020-08-30T04:32:48,588][INFO ][o.e.t.LoggingTaskListener] [esforum] 4688479 finished with response BulkByScrollResponse[took=2.1ms,timed_out=false,sliceId=null,updated=0,created=0,deleted=0,batches=0,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[2020-08-30T04:32:48,652][INFO ][o.e.t.LoggingTaskListener] [esforum] 4688500 finished with response BulkByScrollResponse[took=1.6ms,timed_out=false,sliceId=null,updated=0,created=0,deleted=0,batches=0,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[2020-08-30T04:50:58,797][INFO ][o.e.t.LoggingTaskListener] [esforum] 4690862 finished with response BulkByScrollResponse[took=2.3ms,timed_out=false,sliceId=null,updated=0,created=0,deleted=0,batches=0,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[2020-08-30T04:50:58,834][INFO ][o.e.t.LoggingTaskListener] [esforum] 4690893 finished with response BulkByScrollResponse[took=1.3ms,timed_out=false,sliceId=null,updated=0,created=0,deleted=0,batches=0,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[2020-08-30T04:50:58,887][INFO ][o.e.t.LoggingTaskListener] [esforum] 4690914 finished with response BulkByScrollResponse[took=7ms,timed_out=false,sliceId=null,updated=0,created=0,deleted=0,batches=0,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[2020-08-30T04:57:47,788][INFO ][o.e.t.LoggingTaskListener] [esforum] 4692174 finished with response BulkByScrollResponse[took=2.7ms,timed_out=false,sliceId=null,updated=0,created=0,deleted=0,batches=0,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[2020-08-30T04:57:47,802][INFO ][o.e.t.LoggingTaskListener] [esforum] 4692205 finished with response BulkByScrollResponse[took=1.9ms,timed_out=false,sliceId=null,updated=0,created=0,deleted=0,batches=0,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[2020-08-30T04:57:47,842][INFO ][o.e.t.LoggingTaskListener] [esforum] 4692226 finished with response BulkByScrollResponse[took=1.7ms,timed_out=false,sliceId=null,updated=0,created=0,deleted=0,batches=0,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[2020-08-30T07:32:12,530][INFO ][o.e.t.LoggingTaskListener] [esforum] 4709374 finished with response BulkByScrollResponse[took=1.5ms,timed_out=false,sliceId=null,updated=0,created=0,deleted=0,batches=0,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[2020-08-30T07:32:12,531][INFO ][o.e.t.LoggingTaskListener] [esforum] 4709343 finished with response BulkByScrollResponse[took=3.5ms,timed_out=false,sliceId=null,updated=0,created=0,deleted=0,batches=0,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[2020-08-30T08:43:32,721][INFO ][o.e.t.LoggingTaskListener] [esforum] 4717202 finished with response BulkByScrollResponse[took=3.4ms,timed_out=false,sliceId=null,updated=0,created=0,deleted=0,batches=0,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[2020-08-30T08:43:32,733][INFO ][o.e.t.LoggingTaskListener] [esforum] 4717233 finished with response BulkByScrollResponse[took=2.3ms,timed_out=false,sliceId=null,updated=0,created=0,deleted=0,batches=0,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[2020-08-30T08:44:12,594][INFO ][o.e.t.LoggingTaskListener] [esforum] 4717298 finished with response BulkByScrollResponse[took=1.9ms,timed_out=false,sliceId=null,updated=0,created=0,deleted=0,batches=0,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[2020-08-30T08:44:12,612][INFO ][o.e.t.LoggingTaskListener] [esforum] 4717329 finished with response BulkByScrollResponse[took=1.6ms,timed_out=false,sliceId=null,updated=0,created=0,deleted=0,batches=0,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[2020-08-30T08:44:43,181][INFO ][o.e.t.LoggingTaskListener] [esforum] 4717384 finished with response BulkByScrollResponse[took=1.9ms,timed_out=false,sliceId=null,updated=0,created=0,deleted=0,batches=0,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[2020-08-30T08:44:43,195][INFO ][o.e.t.LoggingTaskListener] [esforum] 4717415 finished with response BulkByScrollResponse[took=1.6ms,timed_out=false,sliceId=null,updated=0,created=0,deleted=0,batches=0,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[2020-08-30T08:45:02,884][INFO ][o.e.t.LoggingTaskListener] [esforum] 4717440 finished with response BulkByScrollResponse[took=58.5ms,timed_out=false,sliceId=null,updated=0,created=0,deleted=0,batches=0,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[2020-08-30T08:47:01,616][INFO ][o.e.t.LoggingTaskListener] [esforum] 4717597 finished with response BulkByScrollResponse[took=2ms,timed_out=false,sliceId=null,updated=0,created=0,deleted=0,batches=0,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[2020-08-30T08:47:01,629][INFO ][o.e.t.LoggingTaskListener] [esforum] 4717628 finished with response BulkByScrollResponse[took=1.4ms,timed_out=false,sliceId=null,updated=0,created=0,deleted=0,batches=0,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[2020-08-30T09:11:36,769][INFO ][o.e.c.m.MetaDataDeleteIndexService] [esforum] [content/5BnvJdR-RD22z2lSm1TmuQ] deleting index
[2020-08-30T09:11:36,864][WARN ][o.e.d.c.m.MetaDataCreateIndexService] [esforum] the default number of shards will change from [5] to [1] in 7.0.0; if you wish to continue using the default of [5] shards, you must manage this on the create index request or with an index template
[2020-08-30T09:11:36,878][INFO ][o.e.c.m.MetaDataCreateIndexService] [esforum] [content] creating index, cause [api], templates [], shards [5]/[1], mappings [content]
 

Et le fichier docker compose utilisé :

version: '3'
services:
  esforum:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.8.10
    container_name: esforum
    environment:
      - node.name=esforum
      - discovery.type=single-node
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms5g -Xmx5g"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - esdata:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
      - 9300:9300
    networks:
      - elastic

volumes:
  esdata:
    driver: local
networks:
  elastic:
    driver: bridge

 

[ForceRouge]

Je pense que le message:

[2020-08-30T01:38:00,006][INFO ][o.e.x.m.a.TransportDeleteExpiredDataAction] [esforum] Deleting expired data

indique une suppression de donnée lié à Machine Learning (ML) uniquement, pas aux data du forum.

Pour moi, ce n'est pas lié au problème que tu rencontres.

Sinon, j'imagine que vous utilisez la licence Basic, auquel cas tu pourrais désactiver la feature ML. Comme ça, t'écarte le problème. En version ES 6.x:

node.ml: false

 

[Cara62]

Merci je vais tester ça.