WSREP multi-master schema database loses data #2820
Description
Question and Steps to reproduce
当前夜莺版本v8.0.0-beta.10，后端使用的数据库架构为PXC架构，使用中发现使用Host类型告警时，偶发性遇到
2025-08-09 01:01:30.644380 ERROR dispatch/consume.go:138 event&{Id:0 Cate:host Cluster: DatasourceId:0 GroupId:1 GroupName:Default Busi Group Hash:63897f82529633cd1928b422956c2819 RuleId:26 RuleName:InstanceDown RuleNote:服务器失联
RuleProd:host RuleAlgo: Severity:1 PromForDuration:60 PromQl: RuleConfig:{"queries":[{"key":"all_hosts","keys":{"labelKey":"","metricKey":"","valueKey":""},"op":"==","values":[]}],"triggers":[{"duration":60,"severity":1,"type":"ta rget_miss"}]} RuleConfigJson:map[queries:[map[key:all_hosts keys:map[labelKey: metricKey: valueKey:] op:== values:[]]] triggers:[map[duration:60 severity:1 type:target_miss]]] PromEvalInterval:60 Callbacks:http:///api/alert/nightingale CallbacksJSON:[http:///api/alert/nightingale] RunbookUrl: NotifyRecovered:1 NotifyChannels: NotifyChannelsJSON:[] NotifyGroups: NotifyGroupsJSON:[] NotifyGroupsObj:[] TargetIdent:g -yjkj-cmcc-gdmz-432 TargetNote: TriggerTime:1754672465 TriggerValue:74 TriggerValues: TriggerValuesJson:{ValuesWithUnit:map[]} Tags:name=target_miss,,ident=g-yjkj-cmcc-gdmz-432,,resourcetype=gstarr,,rulename=InstanceDown TagsJS ON:[name=target_miss ident=g-yjkj-cmcc-gdmz-432 resourcetype=gstarr rulename=InstanceDown] TagsMap:map[name:target_miss ident:g-yjkj-cmcc-gdmz-432 resourcetype:gstarr rulename:InstanceDown] OriginalTags: OriginalTagsJSON:[] Annotations:{} AnnotationsJSON:map[] IsRecovered:false NotifyUsersObj:[] LastEvalTime:1754672465 LastSentTime:1754672465 NotifyCurNumber:1 FirstTriggerTime:1754672465 ExtraConfig:map[enrich_queries:[]] Status:0 Claimant: SubRuleId :0 ExtraInfo:[] Target:0xc001380580 RecoverConfig:{JudgeType:0 RecoverExp:} RuleHash:87b073d21883c459a79c71b913af4ca4 ExtraInfoMap:[] NotifyRuleIDs:[]} persist err:event_persist_check_exists_fail: Error 1047: WSREP has not yet prep ared node for application use rule_id=26 hash=63897f82529633cd1928b422956c2819
导致告警丢失且无法恢复，但是仍会推送webhook
Relevant logs and configurations
2025-08-09 01:01:30.644380 ERROR dispatch/consume.go:138 event&{Id:0 Cate:host Cluster: DatasourceId:0 GroupId:1 GroupName:Default Busi Group Hash:63897f82529633cd1928b422956c2819 RuleId:26 RuleName:InstanceDown RuleNote:服务器失联
RuleProd:host RuleAlgo: Severity:1 PromForDuration:60 PromQl: RuleConfig:{"queries":[{"key":"all_hosts","keys":{"labelKey":"","metricKey":"","valueKey":""},"op":"==","values":[]}],"triggers":[{"duration":60,"severity":1,"type":"ta rget_miss"}]} RuleConfigJson:map[queries:[map[key:all_hosts keys:map[labelKey: metricKey: valueKey:] op:== values:[]]] triggers:[map[duration:60 severity:1 type:target_miss]]] PromEvalInterval:60 Callbacks:api/alert/nightingale CallbacksJSON:[http:///api/alert/nightingale] RunbookUrl: NotifyRecovered:1 NotifyChannels: NotifyChannelsJSON:[] NotifyGroups: NotifyGroupsJSON:[] NotifyGroupsObj:[] TargetIdent:g -yjkj-cmcc-gdmz-432 TargetNote: TriggerTime:1754672465 TriggerValue:74 TriggerValues: TriggerValuesJson:{ValuesWithUnit:map[]} Tags:__name__=target_miss,,ident=g-yjkj-cmcc-gdmz-432,,resourcetype=gstarr,,rulename=InstanceDown TagsJS ON:[__name__=target_miss ident=g-yjkj-cmcc-gdmz-432 resourcetype=gstarr rulename=InstanceDown] TagsMap:map[__name__:target_miss ident:g-yjkj-cmcc-gdmz-432 resourcetype:gstarr rulename:InstanceDown] OriginalTags: OriginalTagsJSON:[] Annotations:{} AnnotationsJSON:map[] IsRecovered:false NotifyUsersObj:[] LastEvalTime:1754672465 LastSentTime:1754672465 NotifyCurNumber:1 FirstTriggerTime:1754672465 ExtraConfig:map[enrich_queries:[]] Status:0 Claimant: SubRuleId :0 ExtraInfo:[] Target:0xc001380580 RecoverConfig:{JudgeType:0 RecoverExp:} RuleHash:87b073d21883c459a79c71b913af4ca4 ExtraInfoMap:[] NotifyRuleIDs:[]} persist err:event_persist_check_exists_fail: Error 1047: WSREP has not yet prep ared node for application use rule_id=26 hash=63897f82529633cd1928b422956c2819
Version
v8.0.0-beta.10
数据库： percona/percona-xtradb-cluster:5.7.29