Continuous Integration repo reminders - revisited

Back in 2017 I wrote about CI reminders. It was specific to 3rd party tools, but it turns out there are other use cases.

It turns out my infrastructure configs have a number of things that I need reminding about.

For web servers I need to know the latest recommendations for SSL configurations. More importantly I need to make the changes and check them.

For terraform I need to review my provider versions and update them.

The original issue - checking 3rd party tools still applies. But clearly there is now a need for a more general solution.

First I made a tool to check and update the repo - ./scripts/periodic:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
#!/bin/bash

export LC_ALL=C 

case "$1" in
  update|check)
    action="$1" ;;
  *)
    echo "ERROR: Usage $0 [check|update]"
    exit 1
    ;;
esac

stamp_expired() {
  local age="$1"
  local stamp="$2"
  if [[ ! -f $stamp ]]; then
    echo 0 > $stamp
  fi
  test $(( $(date +%s) - $(cat $stamp) )) -gt $age
}

stamp_update() {
  date '+%s' > "$1"
}

stamp_snooze() {
  echo $(( $(date '+%s') - $1 )) > "$2"
}

check_thing() {
  local f thing="$1"

  for f in ${thing}_{checking_info,on_update,on_check}; do
    if ! type $f | grep -q 'is a function'; then
      echo "ERROR: missing function $f for $thing."
      exit 1
    fi
  done
  if [[ $age == no ]]; then
    echo "ERROR: Need to set \$age for $thing."
    exit 1
  fi
}

cd "$(git rev-parse --show-toplevel)" || exit 1
for check in .periodic/*.check; do
  thing="${check%.check}"
  thing="${thing#.*/}"
  stamp=".periodic/.${thing}.stamp"
  age=no
  source "$check"
  check_thing $thing
  ${thing}_checking_info
  if stamp_expired $age $stamp; then
    ${thing}_on_$action $stamp
    if [[ $action == check ]]; then
      exit 1
    fi
  fi
done

It’s a bit messy but it looks in .periodic for .check files. Those need to define three functions and a variable called $age.

So from the original example, this is how the 3rd party updater is written - .periodic/3rd.check:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
#!/bin/bash

age=$(( 6 * 30 * 24 * 60 * 60 ))

3rd_checking_info() {
  echo "Checking 3rd party subtree packages..."
}

3rd_on_update() {
  local stamp="$1"

  git=/usr/bin/git

  $git subtree pull -P "roles/src/files/usr/local/src/cvssync" \
       "https://github.com/cvsync/cvsync.git" master
  $git subtree pull -P "roles/src/files/usr/local/src/rcsparse" \
       "https://github.com/corecode/rcsparse.git" master
  $git subtree pull -P "roles/src/files/usr/local/src/cvs2gitdump" \
       "https://github.com/yasuoka/cvs2gitdump.git" master
  $git subtree pull -P "roles/cloud-print/files/usr/local/src/CUPS-Cloud-Print" \
       "git://github.com/simoncadman/CUPS-Cloud-Print.git" tags/20160502
  $git subtree pull -P twitter/ephemeral \
       git@github.com:vickylai/ephemeral.git master

  cat << EOF
Check https://www.niftiestsoftware.com/cups-cloud-print/ to see
if tags/20160502 is the correct current tag. Note that the github
page is currently (1/2017) incorrect on how to install.
EOF

  stamp_update $stamp
}

3rd_on_check() {
  echo "3rd party sources out of date - update with ./scripts/periodic update."
}

This is one that can update the repo automatically once I run ~/scripts/periodic update . Other checks are more manual. The ssl config check for instance - .periodic/ssl.check:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
#!/bin/bash

# Check every 6 months, allow user to snooze.
age=$(( 6 * 30 * 24 * 60 * 60 ))
snooze=$(( $age - 7 * 24 * 60 * 60 ))

ssl_checking_info() {
  echo "Checking ssl cipher configs..."
}

ssl_on_update() {
  local stamp="$1"

  cat << EOF
EXPIRED: It has been six months since you checked ssl configs.

Need to review roles/webfe/files/etc/nginx/include/ssl_settings
and make sure the ssl configurations are correct.  Once that's
done type "yes" to confirm or "snooze" to snooze this alert
for 7 days.
EOF
  read -p "Finished? (yes or snooze): " finished
  case $finished in
    y*)
      stamp_update $stamp ;;
    s*)
      stamp_snooze $snooze $stamp ;;
    *)
      echo "Didn't understand '$finished'. Aborting."
      exit 1
      ;;
  esac
}

ssl_on_check() {
  echo "SSL configs need to be checked."
  echo "Update or snooze with ./scripts/periodic update."
}

This one also offers the option to snooze the alert. Realistically this might fire at a point when I’m busy and can’t make and test SSL config changes. This gives me a quick way to be notified but then snooze the alert to address it later.