Async Queries redis connection error

[MorieeLM]

I have hosted superset on kubernetes and keep getting these errors when trying to generate charts on superset web; the user interface hangs with a display message of "waiting for postgresql" and no charts are gemerated: kubectl logs -n superset -l app=superset-worker --tail=20
Defaulted container "superset" out of: superset, wait-for-postgres-redis (init)
R = retval = fun(*args, kwargs)
File "/app/superset/initialization/init.py", line 114, in call
return task_base.call(self, *args, kwargs)
File "/app/.venv/lib/python3.10/site-packages/celery/app/trace.py", line 736, in protected_call
return self.run(args, **kwargs)
File "/app/superset/tasks/async_queries.py", line 110, in load_chart_data_into_cache
async_query_manager.update_job(
File "/app/superset/async_events/async_query_manager.py", line 306, in update_job
self._cache.xadd(scoped_stream_name, event_data, "", self._stream_limit)
File "/app/superset/async_events/cache_backend.py", line 71, in xadd
return self._cache.xadd(stream_name, event_data, event_id, maxlen)
File "/app/.venv/lib/python3.10/site-packages/redis/commands/core.py", line 3534, in xadd
return self.execute_command("XADD", name, *pieces)
File "/app/.venv/lib/python3.10/site-packages/redis/client.py", line 1266, in execute_command
conn = self.connection or pool.get_connection(command_name, **options)
File "/app/.venv/lib/python3.10/site-packages/redis/connection.py", line 1461, in get_connection
connection.connect()
File "/app/.venv/lib/python3.10/site-packages/redis/connection.py", line 713, in connect
raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 111 connecting to localhost:6379. Connection refused.

please assist with a workaround for this.

I have configOverrides defined as follows: # Proxy and feature flags
ENABLE_PROXY_FIX = True
FEATURE_FLAGS = {
"DASHBOARD_NATIVE_FILTERS": True,
"DASHBOARD_CROSS_FILTERS": True,
"GLOBAL_ASYNC_QUERIES": True,
}

# Celery configuration (don't override the default one, just update it)
CELERY_CONFIG = {
    'broker_url': f'redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_CELERY_DB}',
    'result_backend': f'redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_CELERY_DB}',
    'imports': [
        'superset.sql_lab',
        'superset.tasks.scheduler',
    ],
    'worker_prefetch_multiplier': 10,
    'task_acks_late': True,
    'task_annotations': {
        'sql_lab.get_sql_results': {
            'rate_limit': '100/s',
        },
    }
}

# Cache configuration
CACHE_CONFIG = {
    'CACHE_TYPE': 'RedisCache',
    'CACHE_DEFAULT_TIMEOUT': 300,
    'CACHE_KEY_PREFIX': 'superset_',
    'CACHE_REDIS_HOST': REDIS_HOST,
    'CACHE_REDIS_PORT': REDIS_PORT,
    'CACHE_REDIS_DB': REDIS_DB,
    'CACHE_REDIS_URL': f'redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_DB}',
}

RESULTS_BACKEND = RedisCache(
    host=REDIS_HOST,
    port=REDIS_PORT,
    db=REDIS_DB,
    key_prefix='superset_results'
)

# Global async queries configuration
GLOBAL_ASYNC_QUERIES_JWT_SECRET = os.environ.get('GLOBAL_ASYNC_QUERIES_JWT_SECRET')
GLOBAL_ASYNC_QUERIES_REDIS_CONFIG = {
    'host': REDIS_HOST,
    'port': REDIS_PORT,
    'db': REDIS_DB,
}
GLOBAL_ASYNC_QUERIES_TRANSPORT = 'redis'
GLOBAL_ASYNC_QUERIES_REDIS_STREAM_PREFIX = 'async-events-'
GLOBAL_ASYNC_QUERIES_REDIS_STREAM_LIMIT = 1000
GLOBAL_ASYNC_QUERIES_REDIS_STREAM_LIMIT_FIREHOSE = 100

[dosubot[bot]]

Hi @MorieeLM! I'm Dosu and I’m helping the superset team.

Your error happens because Superset is trying to connect to Redis at localhost:6379, but in Kubernetes, Redis usually runs in its own pod and should be accessed via its service name, not localhost. This causes connection refused errors unless Redis is running in the same container, which is not typical for Kubernetes setups. The UI hangs with "waiting for postgresql" because async queries and Celery workers can't connect to Redis, so queries never complete and results aren't cached or returned.

To fix this, make sure your REDIS_HOST environment variable is set to your Redis service name (for example, superset-redis-master or superset-redis-headless), not localhost. You can check the current value by running kubectl exec on a Superset pod and running env | grep REDIS. Update your Helm values.yaml or Kubernetes manifests to set REDIS_HOST using extraEnv or extraSecretEnv, and reference it in your configOverrides. For example:

extraEnv:
  REDIS_HOST: superset-redis-master
  REDIS_PORT: "6379"
  REDIS_DB: "0"
  REDIS_CELERY_DB: "1"

Then, in your configOverrides, use os.environ.get("REDIS_HOST") and similar for all Redis-related config. After updating, redeploy Superset and the workers so the new settings take effect. If Redis requires authentication, include the username and password in your Redis URLs.

References:

• Kubernetes Redis config best practices
• Superset async queries and Celery config
• How async queries use Redis config

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[dosubot[bot]]

If Superset is still trying to connect to Redis at localhost:6379, even though you set the correct extraEnv, the most common cause is that your configOverrides (the Python code in superset_config.py) is not actually referencing the environment variables at runtime. In Kubernetes, setting extraEnv only makes the variables available to the container—they must also be explicitly read in your configOverrides using os.environ.get or os.getenv. If you use just REDIS_HOST in your configOverrides without os.environ.get, it will default to 'localhost' unless you define REDIS_HOST earlier in the file.

Double-check that your configOverrides look like this:

import os

REDIS_HOST = os.environ.get("REDIS_HOST", "localhost")
REDIS_PORT = os.environ.get("REDIS_PORT", "6379")
REDIS_DB = os.environ.get("REDIS_DB", "1")
REDIS_CELERY_DB = os.environ.get("REDIS_CELERY_DB", "0")

CELERY_CONFIG = {
    'broker_url': f'redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_CELERY_DB}',
    'result_backend': f'redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_CELERY_DB}',
    # ...
}

CACHE_CONFIG = {
    'CACHE_TYPE': 'RedisCache',
    'CACHE_DEFAULT_TIMEOUT': 300,
    'CACHE_KEY_PREFIX': 'superset_',
    'CACHE_REDIS_HOST': REDIS_HOST,
    'CACHE_REDIS_PORT': REDIS_PORT,
    'CACHE_REDIS_DB': REDIS_DB,
    'CACHE_REDIS_URL': f'redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_DB}',
}

If you reference REDIS_HOST directly without assigning it from os.environ.get, Python will not know its value and will fall back to the default in the main config, which is localhost. This is a subtle but common pitfall in Kubernetes deployments with Superset [docs].

If you have already done this and it still fails, check that the environment variables are present in the running pod (kubectl exec <pod> -- env | grep REDIS) and that you can reach Redis from the pod (kubectl exec <pod> -- nc -vz superset-redis-master 6379). If DNS is an issue, try using the Redis service's IP address instead of the DNS name as a test [troubleshooting].

If Redis requires a password or SSL, make sure those are included in your config as well. Let me know if you want a checklist for network troubleshooting or more details on the configOverrides pattern.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[MorieeLM]

@dosu
I did, here is the whole configOverrides section: configOverrides:

Single comprehensive configuration to avoid conflicts

superset_config: |
import os
from cachelib.redis import RedisCache

# Secret key
SECRET_KEY = os.environ.get('SECRET_KEY')

# Database configuration
SQLALCHEMY_DATABASE_URI = os.environ.get('SQLALCHEMY_DATABASE_URI')

# Redis configuration

REDIS_HOST = os.environ.get('REDIS_HOST', 'superset-redis-master')
REDIS_PORT = int(os.environ.get('REDIS_PORT', '6379'))
REDIS_DB = int(os.environ.get('REDIS_DB', '1'))
REDIS_CELERY_DB = int(os.environ.get('REDIS_CELERY_DB', '0'))

# Proxy and feature flags
ENABLE_PROXY_FIX = True
FEATURE_FLAGS = {
    "DASHBOARD_NATIVE_FILTERS": True,
    "DASHBOARD_CROSS_FILTERS": True,
    "GLOBAL_ASYNC_QUERIES": True,
}

# Celery configuration (don't override the default one, just update it)
CELERY_CONFIG = {
    'broker_url': f'redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_CELERY_DB}',
    'result_backend': f'redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_CELERY_DB}',
    'imports': [
        'superset.sql_lab',
        'superset.tasks.scheduler',
    ],
    'worker_prefetch_multiplier': 10,
    'task_acks_late': True,
    'task_annotations': {
        'sql_lab.get_sql_results': {
            'rate_limit': '100/s',
        },
    }
}

# Cache configuration
CACHE_CONFIG = {
    'CACHE_TYPE': 'RedisCache',
    'CACHE_DEFAULT_TIMEOUT': 300,
    'CACHE_KEY_PREFIX': 'superset_',
    'CACHE_REDIS_HOST': REDIS_HOST,
    'CACHE_REDIS_PORT': REDIS_PORT,
    'CACHE_REDIS_DB': REDIS_DB,
    'CACHE_REDIS_URL': f'redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_DB}',
}

RESULTS_BACKEND = RedisCache(
    host=REDIS_HOST,
    port=REDIS_PORT,
    db=REDIS_DB,
    key_prefix='superset_results'
)

# Global async queries configuration
GLOBAL_ASYNC_QUERIES_JWT_SECRET = os.environ.get('GLOBAL_ASYNC_QUERIES_JWT_SECRET')
GLOBAL_ASYNC_QUERIES_REDIS_CONFIG = {
    'host': REDIS_HOST,
    'port': REDIS_PORT,
    'db': REDIS_DB,
}
GLOBAL_ASYNC_QUERIES_TRANSPORT = 'redis'
GLOBAL_ASYNC_QUERIES_REDIS_STREAM_PREFIX = 'async-events-'
GLOBAL_ASYNC_QUERIES_REDIS_STREAM_LIMIT = 1000
GLOBAL_ASYNC_QUERIES_REDIS_STREAM_LIMIT_FIREHOSE = 100

[dosubot[bot]]

Your configOverrides are almost correct, but Superset 5.0+ now requires a new setting for async queries: you must define GLOBAL_ASYNC_QUERIES_CACHE_BACKEND in your superset_config.py. If this is missing, Superset falls back to defaults (localhost:6379), which causes the connection errors you're seeing—even if your environment variables and other configs are set properly. This is a recent change and isn't reflected in older docs or most support threads.

Add the following to your configOverrides, using your existing Redis variables:

GLOBAL_ASYNC_QUERIES_CACHE_BACKEND = {
    'CACHE_TYPE': 'RedisCache',
    'CACHE_REDIS_HOST': REDIS_HOST,
    'CACHE_REDIS_PORT': REDIS_PORT,
    'CACHE_REDIS_DB': REDIS_DB,
    'CACHE_KEY_PREFIX': 'superset_async_',
}

This will ensure async queries use your intended Redis host and DB, and prevent fallback to localhost. After updating, redeploy all Superset pods (web, worker, etc.) so the new config is loaded everywhere. You can find more details in the recent PR and code changes.

If you still see issues, double-check that the environment variables are present in all pods, and that the correct superset_config.py is being loaded. Also, check the logs for which Redis host is actually being used at runtime.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[MorieeLM]

@dosu

it worked, thank you.

[dosubot[bot]]

You're welcome! Glad it worked. Let me know if you need anything else.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}