Skip to content

clear auto-follow errors on deleting pattern#80544

Merged
idegtiarenko merged 12 commits into
elastic:masterfrom
idegtiarenko:77723_clear_auto_follow_stats_on_delete
Nov 24, 2021
Merged

clear auto-follow errors on deleting pattern#80544
idegtiarenko merged 12 commits into
elastic:masterfrom
idegtiarenko:77723_clear_auto_follow_stats_on_delete

Conversation

@idegtiarenko

@idegtiarenko idegtiarenko commented Nov 9, 2021

Copy link
Copy Markdown
Contributor

This change clears auto-follow errors that belongs to patterns that are removed by cluster update.

Closes: #77723

@idegtiarenko idegtiarenko added >bug :Distributed/CCR Issues around the Cross Cluster State Replication features v8.1.0 labels Nov 9, 2021
@elasticmachine elasticmachine added the Team:Distributed Meta label for distributed team. label Nov 9, 2021
@elasticmachine

Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-distributed (Team:Distributed)

);

// then stats are removed as well
assertThat(autoFollowCoordinator.getStats().getRecentAutoFollowErrors().keySet(), empty());

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could check that unrelated errors are not removed

)
);
// and applied
assertThat(autoFollowCoordinator.getStats().getRecentAutoFollowErrors(), hasKey("pattern1:index1"));

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was surprised that the second AutoFollowResult cleared the error from the first one.
Not sure if this is intended behavior.

@henningandersen henningandersen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a few smaller comments, otherwise looking good.

@henningandersen henningandersen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a few comments, will review the latest addition shortly.

}
assert assertNoOtherActiveAutoFollower(newAutoFollowers);
this.autoFollowers = autoFollowers.copyAndPutAll(newAutoFollowers).copyAndRemoveAll(removedRemoteClusters);
this.patterns = Set.copyOf(autoFollowMetadata.getPatterns().keySet());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is too late, it should preferably go before starting any new auto-followers. Otherwise, we risk a race where the auto-follower completes and updates stats before we get here (would have to be twice to become a problem but anyway we can fix this by assigning this earlier).

Comment on lines +1959 to +1960
assertThat(before.getRecentAutoFollowErrors().keySet(), contains("pattern1", "pattern2", "pattern3"));
assertThat(after.getRecentAutoFollowErrors().keySet(), allOf(contains("pattern1", "pattern2"), not(contains("pattern3"))));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is slightly confusing. contains verifies same length and same order, which gives rise to two comments:

  1. I think the order is not relevant. So equalTo a Set.of would be better here, alternatively containsInAnyOrder. The test only passes (I think) because of the use of LinkedHashMap internally.
  2. The not(contains("pattern3")) is true because it is not a list containing only that one element.

I think we should simply use equalTo instead.

I think this comment is true for the other contains verifications below in this PR.

@henningandersen henningandersen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor nit here, otherwise this looks good.

@henningandersen henningandersen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

}

this.patterns = Set.copyOf(autoFollowMetadata.getPatterns().keySet());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should set the patterns to the empty list above when autoFollowMetadata == null too (though I think we never go back to null, so mostly for completeness).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we are also not clearing/stopping autoFollowers once metadata is null. If this happens for whatever reason then it might result in auto-followers still running but not producing any stats

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, perhaps we can simply add an assert before the return above to express the intent here?

# Conflicts:
#	x-pack/plugin/ccr/src/main/java/org/elasticsearch/xpack/ccr/action/AutoFollowCoordinator.java
@idegtiarenko idegtiarenko merged commit 0f10825 into elastic:master Nov 24, 2021
@idegtiarenko idegtiarenko deleted the 77723_clear_auto_follow_stats_on_delete branch November 24, 2021 11:19
idegtiarenko added a commit to idegtiarenko/elasticsearch that referenced this pull request Nov 24, 2021
idegtiarenko added a commit to idegtiarenko/elasticsearch that referenced this pull request Nov 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>bug :Distributed/CCR Issues around the Cross Cluster State Replication features Team:Distributed Meta label for distributed team. v7.16.0 v8.0.0-rc2 v8.1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CCR clear recent auto-follow errors

4 participants