How to Rebuild Search Index Without Downtime

Customer Managed Applies to customer-managed instances of Alation

Applies from version 2023.3.5

Warning

  • Initiate these steps only when directed.

  • The steps require backend access to the Alation server.

Rebuilding the search index operates in the background and does not require downtime. The process duration depends on your Alation database size and may extend to several hours on large instances.

To rebuild the search index:

  1. Use SSH to connect to your Alation server.

  2. The process is designed to be interruption-free, but given its potential length, we recommend using a session management tool like Screen for tracking it. For example, start a Screen session:

    screen -SL rebuild_es_index

  3. Enter the Alation shell.

    sudo /etc/init.d/alation shell

  4. Switch to the alation user.

    sudo su alation

  5. Start the index rebuild with the following script.

    python /opt/alation/django/rosemeta/one_off_scripts/search_rebuild_index.pyc --using default --using default_large

    The script runs in the background. A completion message will appear upon success:

    "message": "[Job <64>] Updating status: NA --> SUCCEEDED", "timestamp": "2024-02-01T02:17:13.509146Z",
"level": "INFO"}[Job <64>] Updating status: NA --> SUCCEEDED

  6. Verify indices with the following command:

    curl localhost:9200/_cat/indices?v

    Look for indices prefixed with second_ in the output. This means that the new indices have been created successfully.

    ../../_images/AdminHowTosRebuildIndexVerify.png

  7. Update index aliases with the next command. Follow the prompts, typing y and pressing Enter to confirm each action. See an example below.

    python /opt/alation/django/rosemeta/one_off_scripts/search_alias_update.pyc

    Example

    Are you sure you want to reassign search index aliases (y/n)?y
GET http://localhost:9200/_alias [status:200 duration:0.003s]
Are you sure you want to reassign search index alias: live (y/n)?y
POST http://localhost:9200/_aliases [status:200 duration:0.011s]

  8. On completion, exit from the alation user and the Alation shell by using the command exit two times.