So while I was integrating LDAP with OpenShift, I had a funny issue - it didn't work. The error message given at the Web UI was also useless, probably for the best, don't want to show how the sausage is made. On top of that, normal logging such as /var/log/{audit/*,messages,secure} showed nothing of these authentication failures.
Come to find out, the normal log-level of OpenShift is pretty low - understandably so, there are a metric TON of events going on in a larger Kubernetes system. So in order to find logging for LDAP and other authentication events, we have to bump up the log level a bit.
1. Download the master-loglevel.yml Ansible Playbook
Red Hat has a handy-dandy Ansible playbook that will modify and collect the log levels of your OpenShift masters!
$ wget https://access.redhat.com/sites/default/files/attachments/master-loglevel.yml
Moving on in the execution of this playbook, it's assumed that your host inventory is located at the default /etc/ansible/hosts location.
2. Increase the log
$ ansible-playbook master-loglevel.yml -t loglevel -e debug_loglevel=6
Now that we're logging at a loud enough volume, let's go and do things that break and don't work and stuff.
3. ?????
Really, at this point I'm not sure what you'd want to look for, but do the things that don't work. For me, I attempted logging into the LDAP users which generated an error which is now logged.
4. PROFIT!!!!1
Let's pull in those juicy logs:
$ ansible-playbook -i inventory master-loglevel.yml -t collect
Logs are saved under /tmp/master-logs/ of your ansible host, for each master .tar.gz.
It contains OpenShift Master API, Controllers and ETCD logs.
Odds are, you want to take a look at those OpenShift Master API logs.
5. Save dat cheddar (and log space)
Don't forget to turn off the logging to something more sane and less verbose...
$ ansible-playbook master-loglevel.yml -t loglevel -e debug_loglevel=2
Hopefully, this helps you track down some of those more system-level issues that you may face as an SRE!
Top comments (0)