Skip to content

[ML] ML legacy index templates that are no longer needed should be deleted#80874

Merged
droberts195 merged 2 commits into
elastic:7.16from
droberts195:delete_ml_legacy_templates_716
Nov 19, 2021
Merged

[ML] ML legacy index templates that are no longer needed should be deleted#80874
droberts195 merged 2 commits into
elastic:7.16from
droberts195:delete_ml_legacy_templates_716

Conversation

@droberts195

@droberts195 droberts195 commented Nov 19, 2021

Copy link
Copy Markdown

We no longer need any ML legacy index templates, as we've switched to
either composable index templates or system indices. All the ML legacy
templates we've created over the years should be deleted, as they're
just confusing clutter within the cluster now.

This is the 7.16 version of #80876.

…leted

We no longer need any ML legacy index templates, as we've switched to
either composable index templates or system indices. All the ML legacy
templates we've created over the years should be deleted, as they're
just confusing clutter within the cluster now.
@droberts195 droberts195 added >bug :ml Machine learning v7.16.1 labels Nov 19, 2021
@droberts195 droberts195 marked this pull request as ready for review November 19, 2021 15:29
@elasticmachine elasticmachine added the Team:ML Meta label for the ML team label Nov 19, 2021
@elasticmachine

Copy link
Copy Markdown
Collaborator

Pinging @elastic/ml-core (Team:ML)

@dimitris-athanasiou dimitris-athanasiou left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment on lines +162 to +165
// Don't delete the legacy templates until the entire cluster is on a version that supports composable templates
if (state.nodes().getMinNodeVersion().before(MlIndexTemplateRegistry.COMPOSABLE_TEMPLATE_SWITCH_VERSION)) {
return true;
}

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This bit is the key difference in the 7.16 branch compared to #80876.

@droberts195 droberts195 merged commit 9d99587 into elastic:7.16 Nov 19, 2021
@droberts195 droberts195 deleted the delete_ml_legacy_templates_716 branch November 19, 2021 16:31
droberts195 added a commit to droberts195/elasticsearch that referenced this pull request Jan 6, 2022
In elastic#80874 ML legacy templates were removed by continuously
monitoring the cluster state and removing the templates once
all nodes were on version 7.14 or higher.

For 8.x this is unnecessarily complex. We know the oldest
possible node in the cluster will be 7.17, so can just remove
all ML legacy templates from the cluster state unconditionally
on startup.
droberts195 added a commit that referenced this pull request Jan 6, 2022
In #80874 ML legacy templates were removed by continuously
monitoring the cluster state and removing the templates once
all nodes were on version 7.14 or higher.

For 8.x this is unnecessarily complex. We know the oldest
possible node in the cluster will be 7.17, so can just remove
all ML legacy templates from the cluster state unconditionally
on startup.
astefan pushed a commit to astefan/elasticsearch that referenced this pull request Jan 7, 2022
In elastic#80874 ML legacy templates were removed by continuously
monitoring the cluster state and removing the templates once
all nodes were on version 7.14 or higher.

For 8.x this is unnecessarily complex. We know the oldest
possible node in the cluster will be 7.17, so can just remove
all ML legacy templates from the cluster state unconditionally
on startup.
astefan pushed a commit to astefan/elasticsearch that referenced this pull request Jan 7, 2022
In elastic#80874 ML legacy templates were removed by continuously
monitoring the cluster state and removing the templates once
all nodes were on version 7.14 or higher.

For 8.x this is unnecessarily complex. We know the oldest
possible node in the cluster will be 7.17, so can just remove
all ML legacy templates from the cluster state unconditionally
on startup.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>bug :ml Machine learning Team:ML Meta label for the ML team v7.16.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants