Skip to content

Commit 382e98b

Browse files
earthgeckoearthgecko
authored
Merge pull request #113 from earthgecko/luminosity
v1.2.17
2 parents 0f9ad50 + 60c64db commit 382e98b

File tree

11 files changed

+353
-16
lines changed

11 files changed

+353
-16
lines changed

‎docs/releases/1_2_16.rst

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,9 @@ Update notes
2424
- There are changes to the DB in v1.2.16
2525
- There are changes to settings.py in v1.2.16
2626

27+
.. note: the requirements.txt specifies urllib3>=1.25.1 however requests needs
28+
this fix at <=1.24.3. Please amend the requirements.txt
29+
2730
How to update from v1.2.15
2831
--------------------------
2932

‎docs/releases/1_2_17.rst

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
======
2+
1.2.17
3+
======
4+
5+
v1.2.17 - May 15, 2019
6+
7+
This is a minor bugs fix release and allowing for the use of hosted slack
8+
accounts and public slack channels.
9+
10+
Changes in v1.2.17
11+
------------------
12+
13+
- luminosity - do not analyse short time series. Only return a time series
14+
sample if the sample has sufficient data points otherwise get_anomalies() will
15+
throw and error (3008)
16+
- Ionosphere - could not determine values from metrics_db_object. Handle if the
17+
metric_ionosphere_enabled value cannot be determined (2984)
18+
- Add metrics id to relevant Ionosphere pages (2990)
19+
- Determine slack group/channel using free or hosted slack account response (2986)
20+
- Determine slack_thread_ts using private and public channel (2986)
21+
- If the Skyline MySQL database is on a remote host 2 seconds in panorama.py
22+
is sometimes not sufficient time to process, increased to 10 (2646)
23+
24+
Update notes
25+
------------
26+
27+
- These update instruction apply to upgrading from v1.2.16 to v1.2.17 only.
28+
However as with all Skyline updates it is possible to go through the update
29+
notes for each version and make your own update notes/process to take you from
30+
version x to version y.
31+
- There are no changes to the DB in v1.2.17
32+
- There are no changes to settings.py in v1.2.17
33+
34+
How to update from v1.2.16
35+
--------------------------
36+
37+
- Download the new release tag or clone/update to get it to a temp location,
38+
ready to be deployed.
39+
40+
.. code-block:: bash
41+
42+
NEW_SKYLINE_VERSION="v1.2.17" # Your new Skyline version
43+
OLD_SKYLINE_VERSION="v1.2.16" # Your old Skyline version
44+
45+
CURRENT_SKYLINE_PATH="/opt/skyline/github/skyline" # Your Skyline path
46+
NEW_SKYLINE_PATH="${CURRENT_SKYLINE_PATH}.${NEW_SKYLINE_VERSION}" # Your new Skyline path
47+
48+
mkdir -p "${CURRENT_SKYLINE_PATH}.${NEW_SKYLINE_VERSION}"
49+
cd "${CURRENT_SKYLINE_PATH}.${NEW_SKYLINE_VERSION}"
50+
git clone https://github.com/earthgecko/skyline .
51+
git checkout "$NEW_SKYLINE_VERSION"
52+
53+
cp "$NEW_SKYLINE_PATH/skyline/settings.py" "$NEW_SKYLINE_PATH/skyline/settings.py.${NEW_SKYLINE_VERSION}.bak"
54+
cat "${CURRENT_SKYLINE_PATH}/skyline/settings.py" > "$NEW_SKYLINE_PATH/skyline/settings.py"
55+
56+
# Stop all other Skyline services
57+
SKYLINE_SERVICES="horizon
58+
analyzer
59+
mirage
60+
crucible
61+
boundary
62+
ionosphere
63+
luminosity
64+
panorama
65+
webapp"
66+
for i in $SKYLINE_SERVICES
67+
do
68+
/etc/init.d/$i stop
69+
done
70+
71+
- Move your current Skyline directory to a backup directory and move the new
72+
Skyline v1.2.17 from the temp location to your working Skyline directory,
73+
(change your paths as appropriate) e.g.
74+
75+
.. code-block:: bash
76+
77+
mv "$CURRENT_SKYLINE_PATH" "${CURRENT_SKYLINE_PATH}.${OLD_SKYLINE_VERSION}"
78+
mv "$NEW_SKYLINE_PATH" "$CURRENT_SKYLINE_PATH"
79+
80+
- Start the all Skyline services (change as appropriate for your set up) e.g.
81+
82+
.. code-block:: bash
83+
84+
# Start all other Skyline services
85+
SKYLINE_SERVICES="panorama
86+
luminosity
87+
horizon
88+
analyzer
89+
mirage
90+
crucible
91+
boundary
92+
ionosphere
93+
webapp"
94+
for i in $SKYLINE_SERVICES
95+
do
96+
/etc/init.d/$i start
97+
done
98+
99+
- Check the logs
100+
101+
.. code-block:: bash
102+
103+
# How are they running
104+
tail -n 20 /var/log/skyline/*.log
105+
106+
# Any errors - each app
107+
find /var/log/skyline -type f -name "*.log" | while read skyline_logfile
108+
do
109+
echo "#####
110+
# Checking for errors in $skyline_logfile"
111+
cat "$skyline_logfile" | grep -B2 -A10 -i "error ::\|traceback" | tail -n 60
112+
echo ""
113+
echo ""
114+
done

‎skyline/analyzer/alerters.py

Lines changed: 86 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1264,18 +1264,102 @@ def alert_slack(alert, metric, context):
12641264
else:
12651265
logger.info('alert_slack - sent slack message')
12661266
if slack_thread_updates:
1267+
# @added 20190508 - Bug #2986: New slack messaging does not handle public channel
1268+
# Issue #111: New slack messaging does not handle public channel
1269+
# The sc.api_call 'files.upload' response which generates
1270+
# slack_file_upload has a different structure depending
1271+
# on whether a channel is private or public. That also
1272+
# goes for free or hosted slack too.
1273+
slack_group = None
1274+
slack_group_list = None
1275+
12671276
# This is basically the channel id of your channel, the
12681277
# name could be used so that if in future it is used or
12691278
# displayed in a UI
1279+
# @modified 20190508 - Bug #2986: New slack messaging does not handle public channel
1280+
# Issue #111: New slack messaging does not handle public channel
1281+
# This block only works for free slack workspace private
1282+
# channels. Although this should be handled in the
1283+
# SLACK_OPTS as slack_account_type: 'free|hosted' and
1284+
# default_channel_type = 'private|public', it is going
1285+
# to be handled in the code for the time being so as not
1286+
# to inconvience users to update their settings.py for
1287+
# v.1.2.17 # TODO next release with settings change add
1288+
# these.
1289+
# try:
1290+
# slack_group = slack_file_upload['file']['groups'][0].encode('utf-8')
1291+
# slack_group_list = slack_file_upload['file']['shares']['private'][slack_group]
1292+
# slack_thread_ts = slack_group_list[0]['ts'].encode('utf-8')
1293+
# logger.info('alert_slack - slack group is %s and the slack_thread_ts is %s' % (
1294+
# str(slack_group), str(slack_thread_ts)))
1295+
# except:
1296+
# logger.info(traceback.format_exc())
1297+
# logger.error('error :: alert_slack - faied to determine slack_thread_ts')
1298+
1299+
slack_group = None
1300+
slack_group_trace_groups = None
1301+
slack_group_trace_channels = None
12701302
try:
12711303
slack_group = slack_file_upload['file']['groups'][0].encode('utf-8')
1304+
logger.info('alert_slack - slack group has been set from \'groups\' as %s' % (
1305+
str(slack_group)))
12721306
slack_group_list = slack_file_upload['file']['shares']['private'][slack_group]
12731307
slack_thread_ts = slack_group_list[0]['ts'].encode('utf-8')
12741308
logger.info('alert_slack - slack group is %s and the slack_thread_ts is %s' % (
12751309
str(slack_group), str(slack_thread_ts)))
12761310
except:
1277-
logger.info(traceback.format_exc())
1278-
logger.error('error :: alert_slack - faied to determine slack_thread_ts')
1311+
slack_group_trace_groups = traceback.format_exc()
1312+
logger.info('alert_slack - failed to determine slack_group using groups')
1313+
if not slack_group:
1314+
# Try by channel
1315+
try:
1316+
slack_group = slack_file_upload['file']['channels'][0].encode('utf-8')
1317+
logger.info('alert_slack - slack group has been set from \'channels\' as %s' % (
1318+
str(slack_group)))
1319+
except:
1320+
slack_group_trace_channels = traceback.format_exc()
1321+
logger.info('alert_slack - failed to determine slack_group using channels')
1322+
logger.error('error :: alert_slack - failed to determine slack_group using groups or channels')
1323+
logger.error('error :: alert_slack - traceback from slack_group_trace_groups follows:')
1324+
logger.error(str(slack_group_trace_groups))
1325+
logger.error('error :: alert_slack - traceback from slack_group_trace_channels follows:')
1326+
logger.error(str(slack_group_trace_channels))
1327+
logger.error('error :: alert_slack - faied to determine slack_thread_ts')
1328+
slack_group_list = None
1329+
if slack_group:
1330+
slack_group_list_trace_private = None
1331+
slack_group_list_trace_public = None
1332+
# Try private channel
1333+
try:
1334+
slack_group_list = slack_file_upload['file']['shares']['private'][slack_group]
1335+
logger.info('alert_slack - slack_group_list determined from private channel and slack_group %s' % (
1336+
str(slack_group)))
1337+
except:
1338+
slack_group_list_trace_private = traceback.format_exc()
1339+
logger.info('alert_slack - failed to determine slack_group_list using private channel')
1340+
if not slack_group_list:
1341+
# Try public channel
1342+
try:
1343+
slack_group_list = slack_file_upload['file']['shares']['public'][slack_group]
1344+
logger.info('alert_slack - slack_group_list determined from public channel and slack_group %s' % (
1345+
str(slack_group)))
1346+
except:
1347+
slack_group_list_trace_public = traceback.format_exc()
1348+
logger.info('alert_slack - failed to determine slack_group_list using public channel')
1349+
logger.info('alert_slack - failed to determine slack_group_list using private or public channel')
1350+
logger.error('error :: alert_slack - traceback from slack_group_list_trace_private follows:')
1351+
logger.error(str(slack_group_list_trace_private))
1352+
logger.error('error :: alert_slack - traceback from slack_group_list_trace_public follows:')
1353+
logger.error(str(slack_group_list_trace_public))
1354+
logger.error('error :: alert_slack - faied to determine slack_thread_ts')
1355+
if slack_group_list:
1356+
try:
1357+
slack_thread_ts = slack_group_list[0]['ts'].encode('utf-8')
1358+
logger.info('alert_slack - slack group is %s and the slack_thread_ts is %s' % (
1359+
str(slack_group), str(slack_thread_ts)))
1360+
except:
1361+
logger.error(traceback.format_exc())
1362+
logger.info('alert_slack - failed to determine slack_thread_ts')
12791363
else:
12801364
send_text = initial_comment + ' :: error :: there was no graph image to upload'
12811365
send_message = sc.api_call(

‎skyline/ionosphere/ionosphere.py

Lines changed: 17 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1028,23 +1028,32 @@ def engine_disposal(engine):
10281028
# Moved get_metrics_db_object block to common_functions.py
10291029
metrics_db_object = get_metrics_db_object(base_name)
10301030
if metrics_db_object:
1031+
metrics_id = None
10311032
try:
10321033
metrics_id = int(metrics_db_object['id'])
1033-
metric_ionosphere_enabled = int(metrics_db_object['ionosphere_enabled'])
1034-
if metric_ionosphere_enabled is not None:
1035-
training_metric = False
1036-
else:
1037-
training_metric = True
1038-
if metric_ionosphere_enabled == 1:
1039-
training_metric = False
10401034
except:
10411035
# @added 20190509 - Bug #2984: Ionosphere - could not determine values from metrics_db_object
10421036
# Added a traceback here to debug an issue
10431037
logger.error(traceback.format_exc())
1044-
logger.error('error :: could not determine values from metrics_db_object for %s' % base_name)
1038+
logger.error('error :: could not determine id from metrics_db_object for %s' % base_name)
10451039
metrics_id = None
10461040
metric_ionosphere_enabled = None
10471041
training_metric = True
1042+
if metrics_id:
1043+
# @modified 20190510 - Bug #2984: Ionosphere - could not determine values from metrics_db_object
1044+
# metric_ionosphere_enabled = int(metrics_db_object['ionosphere_enabled'])
1045+
metric_ionosphere_enabled = None
1046+
try:
1047+
metric_ionosphere_enabled = int(metrics_db_object['ionosphere_enabled'])
1048+
except:
1049+
logger.error(traceback.format_exc())
1050+
logger.error('error :: could not determine ionosphere_enabled from metrics_db_object for %s' % base_name)
1051+
if metric_ionosphere_enabled is not None:
1052+
training_metric = False
1053+
else:
1054+
training_metric = True
1055+
if metric_ionosphere_enabled == 1:
1056+
training_metric = False
10481057
else:
10491058
metrics_id = None
10501059
metric_ionosphere_enabled = None

‎skyline/luminosity/process_correlations.py

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -194,6 +194,16 @@ def get_anomalous_ts(base_name, anomaly_timestamp):
194194
anomaly_ts.append((int(ts), value))
195195
if int(ts) > anomaly_timestamp:
196196
break
197+
198+
# @added 20190515 - Bug #3008: luminosity - do not analyse short time series
199+
# Only return a time series sample if the sample has sufficient data points
200+
# otherwise get_anomalies() will throw and error
201+
len_anomaly_ts = len(anomaly_ts)
202+
if len_anomaly_ts <= 9:
203+
logger.info('%s insufficient data not retrieved, only %s data points surfaced, not correlating' % (
204+
str(base_name), str(len_anomaly_ts)))
205+
return []
206+
197207
return anomaly_ts
198208

199209

0 commit comments

Comments
 (0)