Monitoring WSO2 Identity Server with Prometheus

Prometheus can be used to monitor the JVM metrics of WSO2 Identity Server. For this, you need to download,

• Prometheus
• Prometheus JMX exporter
• WSO2 Identity Server

First of all, we need to configure JMX exporter to expose the Identity Server metrics. This exporter is intended to be run as a Java Agent, exposing an HTTP server and serving metrics of the Identity Server JVM. We start with creating the config file for the agent. Create a file named is.yml in the bin directory of the Identity Server. In the initial phase, we will stick to basic configs. Add below content to the created yml file.

---
lowercaseOutputLabelNames: true
lowercaseOutputName: true

Now open the wso2server.sh and add the Java Agent. Scroll down to the bottom of the file and you will notice the "-D" parameters added. At the end of those parameters add below parameter also.

-javaagent:$CARBON_HOME/bin/jmx_prometheus_javaagent-0.11.0.jar=8080:$CARBON_HOME/bin/is.yml \

After the config, the file would look like below.

...
do
    $JAVACMD \
    ...
    -Dhttpclient.hostnameVerifier="DefaultAndLocalhost" \
    -javaagent:$CARBON_HOME/bin/jmx_prometheus_javaagent-0.11.0.jar=8080:$CARBON_HOME/bin/is.yml \
    org.wso2.carbon.bootstrap.Bootstrap $*
    status=$?
done

We are finished in the Identity Server side and you can start the identity server.

Now we can start configuring Prometheus. Go the downloaded Prometheus directory. Create a file called is.yml there. Add below content.

# my global config
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
  - static_configs:
    - targets:
      # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:

# A scrape configuration containing exactly one endpoint to scrape:
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'WSO2 Identity Server'

    static_configs:
    - targets: ['localhost:8080']

Now we can start Prometheus. Run below command from the Prometheus directory.

./prometheus --config.file=is.yml

Everything is set. Navigate to the http://localhost:9090/graph to use Prometheus's built-in expression browser. Now you can monitor the JVM metrics. To view memory consumptions of the JVM enter this into the expression console and click "Execute" button.

jvm_memory_pool_bytes_used

You should be able to see the time series graph of the memory usage like below.

We will dig deeper into configurations in a future post.

New Theme for Oh My ZSH

I have been using Oh My ZSH for quite some time now with robbyrussell and agnoster themes. While neither of them was what I was looking for, I couldn't find a similar theme I was thinking about. When I was recently looking for a theme again af-magic caught my eye. While it was not quite what I was looking for, it was the closest I could find.

Then I thought of customizing this theme to match my liking. Starting from the af-magic theme, I created my own theme. I created a file called my-theme.zsh-theme in ~/.oh-my-zsh/custom/themes folder. Added below content to the file.

# my-theme.zsh-theme

if [ $UID -eq 0 ]; then NCOLOR="red"; else NCOLOR="green"; fi
local return_code="%(?..%{$fg[red]%}%? ↵%{$reset_color%})"

# color vars
eval my_gray='$FG[237]'
eval my_orange='$FG[214]'

# primary prompt
PROMPT='$my_gray%n%{$reset_color%}:% $FG[032]%~\
$(git_prompt_info) \
$FG[105]%(!.#.»)%{$reset_color%} '
PROMPT2='%{$fg[red]%}\ %{$reset_color%}'
RPS1='${return_code}'

# git settings
ZSH_THEME_GIT_PROMPT_PREFIX="$FG[075]($FG[078]"
ZSH_THEME_GIT_PROMPT_CLEAN=""
ZSH_THEME_GIT_PROMPT_DIRTY="$my_orange*%{$reset_color%}"
ZSH_THEME_GIT_PROMPT_SUFFIX="$FG[075])%{$reset_color%}"

Then add the below line to the .zshrc file in your home directory and remove the already defined ZSH_THEME variable.

ZSH_THEME="my-theme"

Source the .zshrc file and below is the result.