Ansible

Installing Ansible

Installing

Ansible is a configuration management (CM) tool to repicate and update configurations across servers.

Install WSL in Windows to use Linux:

List available versions: wsl --list --online
Install specific distro: wsl --install Ubuntu-24.04
Restart if prompted and complete first time setup to add user
Check version: wsl -l -v
Set default version: wsl --set-default-version 2

Install Ansible through Linux shell

Install Ansible PPA:

sudo apt install -y software-properties-common
sudo add-apt-repository --yes --update ppa:ansible/ansible

Install Ansible: sudo apt install -y ansible
Check version: ansible --version

Useful

Ansible uses indentation instead of brackets to determine the code level/block
To access files in Windows partition while running WSL Linux shell, use mtn/ to access any file drives

C:\Users\Tran\Desktop translate to /mnt/c/Users/Tran/Desktop in Linux shell

VS Code has extensions that are useful for ansible files

WSL: (ms-vscode-remote.remote-wsl)

Allow VS Code to connect to WSL OS and run command through the Linux shell, mainly python commands
After opening ansible project in VS Code, press Ctrl + Shift + P and type WSL: Reopen Folder in WSL
Allow cmd like ansible-galaxy role init [role-name]
Allow cmd like ansible-playbook -i tests/inventory.ini tests/test.yml

Ansible: (redhat.ansible)

Need to install in VS Code while it's in WSL mode as well
Has autocomplete for modules, syntax validation, and hover a module to see its documentation

YAML: (redhat.vscode-yaml)

Need to install in VS Code while it's in WSL mode as well
Has YAML validation, autocomplete, and built-in Kubernetes syntax support

Setup and Running Ansible

Setting Up Ansible

Ansible, at the bare minimum, requires a inventory file and a playbook.yml file

inventory: holds the group of servers IP and SSH info

[solr-nodes]
{host1} ansible_user={username} ansible_ssh_private_key_file={private key}
{host2} ansible_user={username} ansible_ssh_private_key_file={private key}

[solr-zk]
{host1} ansible_user={username} ansible_ssh_private_key_file={private key}
=================================================
// Alternative

[solr-nodes]
{host1}
{host2}

[solr-nodes:vars]
webhhook="xxx"

[solr-zk]
zk1 ansible_host=10.0.0.5
zk2 ansible_host=10.0.0.6

[solr-zk:vars]
webhhook="xxx"

[all:vars]
ansible_user={username} 
ansible_ssh_private_key_file={private key}

ssh_port=22

[solr-nodes]: the name of the group
ansible_user: SSH username
ansible_ssh_private_key_file: SSH key

playbook.yml: holds the tasks that can be run on any server in inventory

---                    # Indicate start of YAML file
- hosts: solr-nodes    # Group name the tasks are for
  become: true         # Use sudo

  tasks:                         # list of actions
    - name: Install nginx        # description 
      apt:                       # ansible module
        name: nginx
        state: present
        update_cache: yes

Running Ansible

Prep:

Export the env variable: export ANSIBLE_HOST_KEY_CHECKING=False so ansible would accept password protected ssh key
If ssh key is on Windows, then copy it over to WSL system at "~/.ssh/" and change permission to 600
Ansible does not allow ssh login as root, so a user need to be created on the server beforehand

sudo adduser [user]
sudo usermod -aG sudo [user]
sudo apt update && sudo apt upgrade -y: apt should also be updated and upgraded
sudo apt install python3-passlib -y: to hash password in vault

To run the playbook:

ansible-playbook -i inventory playbook.yml
ansible-playbook -i inventory playbook.yml --check --diff: dry run that show the exact file differences
ansible-playbook playbook.yml --ask-vault-pass: run playbook with value secrets inserted
ansible-playbook -i inventory playbook.yml -u [username] --private-key [private key] --ask-pass -K: ask for sudo password

ansible-playbook -i inventory playbook.yml --limit zk1,zk2: limit tasks to zk1 and zk2 servers label in inventory
ansible-playbook -i inventory playbook.yml --limit 10.0.0.5: limit task to specific IP

ansible-playbook -i inventory playbook.yml --ask-vault-pass -K --check --diff --limit zk2: main check
ansible-playbook -i inventory playbook.yml --ask-vault-pass -K --limit zk2: main overall

Directory Structure

In an ansible folder, the roles/ directory is used to organize reusable server configuration

defaults/: contain the default variables as backup if they were not specified
tasks/: the main playbook
files/: static files copied to servers
templates/: Jinja2 templates (dynamic configs)
handlers/: Actions triggered when something changes (restart nginx etc.)
vars/: Variables specific to the role

ansible/
├── group_vars/all/vault.yml
├── inventory
├── playbook.yml
└── roles/
    ├── nginx/
    │   ├── tasks/
    │   │   └── main.yml
    │   ├── files/
    │   │   └── nginx.conf
    │   ├── templates/
    │   │   └── nginx.conf.j2
    │   ├── handlers/
    │   │   └── main.yml
    │   └── vars/
    │       └── main.yml
    │
    └── solr/
        ├── tasks/
        │   └── main.yml
        ├── files/
        │   └── solr.xml
        └── templates/

Using roles in playbook

---
- hosts: solr-nodes
  become: true

  roles:
    - nginx
    - solr         
============================================================================
Ansible automatically loads:
    roles/nginx/tasks/main.yml
    roles/solr/tasks/main.yml

Defaults Directory

The defaults/ folder has a main.yml file, containing a list of default variables for the role; they are backup if these parameters were not specified

# defaults/main.yml
ssh_port: 22
solr_port: 8983

Tasks Directory

The tasks/ folder has a main.yml file, containing a list of tasks that get inserted into the playbook

# tasks/main.yml
---
- name: Install nginx
  apt:
    name: nginx
    state: present
    update_cache: yes

- name: Enable nginx
  service:
    name: nginx
    state: started
    enabled: true

Files Directory

The files/ is for static files you want to copy exactly as-is to the remote server

The copy module replace the file at the destination if it exists
Numeric(Octal) Mode for File Permission:

7: rwx
6: rw-
5: r-x
4: r--
3: -wx
2: -w-
1: --x
0: ---

roles/
└── nginx/
    ├── tasks/
    │   └── main.yml
    ├── files/
    │   ├── mysite.conf
    │   └── ssl/
    │       └── example.crt     
============================================================================
# Copy single file
- name: Copy SSL key
  copy:
    src: ssl/example.key
    dest: /etc/ssl/private/example.key
    owner: root
    group: root
    mode: '0600'
============================================================================
# Copy multiple files
- name: Copy multiple static files
  copy:
    src: "{{ item.src }}"
    dest: "{{ item.dest }}"
    owner: root
    group: root
    mode: "{{ item.mode }}"
  loop:
    - { src: "mysite.conf", dest: "/etc/nginx/sites-available/mysite.conf", mode: "0644" }
    - { src: "ssl/example.crt", dest: "/etc/ssl/certs/example.crt", mode: "0644" }
    - { src: "ssl/example.key", dest: "/etc/ssl/private/example.key", mode: "0600" }

Templates Directory

The templates/ is used for dynamic configuration files that need variable substitution or logic. It uses Jinja2, .j2, syntax.

Jinja2 also supports conditionals in the templates
template module is the same as copy module except used for .j2 file
Facts: host-specific variables that Ansible automatically knows and replace

ansible_host: IP/hostname Ansible connects to (10.0.0.5)
inventory_hostname: The name used in the inventory (zk1)
ansible_user: SSH user connecting
ansible_port: SSH port
ansible_fqdn: Fully qualified domain name
ansible_hostname: Hostname
ansible_default_ipv4.address: Primary IPv4 address

roles/
└── nginx/
    ├── tasks/
    │   └── main.yml
    ├── templates/
    │   └── nginx.conf.j2
    └── handlers/
        └── main.yml
============================================================================
# nginx.conf.j2
server {
    listen 80;
    server_name {{ server_name }};          # {{server_name}} will be replaced

    {% if enable_ssl %}                     # start if
    listen 443 ssl;                         |
    ssl_certificate {{ ssl_cert }};         |
    ssl_certificate_key {{ ssl_key }};      |
    {% endif %}                             # end if

    root {{ document_root }};
}  
============================================================================
# playbook.yml
---
- hosts: webservers
  become: true
  
  vars:
    server_name: www.example.com            # variable replacement
    document_root: /var/www/html
    enable_ssl: true                        # conditional variable
    ssl_cert: /etc/ssl/certs/example.crt
    ssl_key: /etc/ssl/private/example.key      
 
  roles:
    - nginx
============================================================================
# nginx/tasks/main.yml
---
- name: Render nginx config from template
  template:                          # How to copy templates to dest
    src: nginx.conf.j2
    dest: /etc/nginx/sites-available/mysite.conf
    owner: root
    group: root
    mode: '0644'
  notify: restart nginx

Vars Directory

The main.yml file whole the variables that can be applied to the whole role

Good for variables that rarely change

# vars/main.yml
---
nginx_package: nginx
nginx_port: 80
document_root: /var/www/html
ssl_cert: /etc/ssl/certs/example.crt
ssl_key: /etc/ssl/private/example.key

Handlers Directory

The handlers/ directory in Ansible roles is used for actions that should run only when something changes

Commonly used to restart services | reload service | restart daemons after config changes
Handlers are run AFTER all tasks in the playbook finish so even if a handler is triggered multiple times, it'll only run once

# This task will force any pending handlers to run immediately
- name: Flush handlers to restart Solr before proceeding
  meta: flush_handlers

# handlers/main.yml
---
- name: restart nginx
  service:
    name: nginx
    state: restarted
============================================================================     
# nginx/tasks/main.yml
---
- name: Install nginx               # handler alias must match value of notify parameter
  apt:
    name: nginx
    state: present
    
- name: Upload nginx config
  template:
    src: nginx.conf.j2
    dest: /etc/nginx/nginx.conf
  notify: restart nginx             # if nginx.conf was changed then handler "restart nginx" will run

- name: Upload site config
  template:
    src: site.conf.j2
    dest: /etc/nginx/sites-enabled/site.conf
  notify: restart nginx             # if site.conf was changed then handler "restart nginx" will run

Modules

Ansible modules are standalone scripts that perform the tasks

Index of Modules: link
state parameter: tells Ansible the desired end condtion of a resource
loop: keyword used to repeat a task multiple times while replacing "{{ item }}" each time

Can also be used to loop over a range

- name: Upload config files
  copy:
    src: "{{ item.src }}"
    dest: "{{ item.dest }}"
  loop:
    - { src: "nginx.conf", dest: "/etc/nginx/nginx.conf" }
    - { src: "site.conf", dest: "/etc/nginx/sites-enabled/site.conf" }

become: true: run task/playbook as sudo
vars: set variable block for task/playbook level
register: [variable]: store the result of the task into the variable

[variable].stdout: output what was stored
[variable].stdout_lines: output what was stored into an array/ as multi-lines

when: [variable].changed: run the task only if the registered variable's task was changed
ignore_errors: "{{ ansible_check_mode | default(false) }}": ignore the task if it gave error during --check

apt: install/remove/update packages

state: present: ensure package installed or file is there
state: absent: ensure package or file is removed
state: lastest: upgrade package or file to newest version
state: reinstalled: force reinstall

- name: Install packages
  apt:
    name:
      - nginx
      - openjdk-17-jdk
      - curl
    state: present

service: manage services

state: started: service is running
state: stopped: service is stopped
state: restarted: restart service
state: reloaded: reload config
enabled: true: enable service at boot
disabled: true: disable service at boot

- name: Ensure nginx is running
  service:
    name: nginx
    state: started
    enabled: true

copy: upload static files

state: file: creates or updates the file
state: absent: remove the file
state: directory: creates a directory instead

- name: Upload config file
  copy:
    src: nginx.conf
    dest: /etc/nginx/nginx.conf
    owner: root
    group: root
    mode: "0644"

template: upload dynamic config

state: file: creates or updates the file
state: absent: remove the file

- name: Upload nginx config template
  template:
    src: nginx.conf.j2
    dest: /etc/nginx/nginx.conf

file: manage files and directories

state: directory: creates directory
state: file: creates empty file
state: touch: Updates file timestamp or creates file if missing
state: absent: remove file or directory
state: link: create a symbolic link
state: hard: create a hard link

- name: Create Solr directory
  file:
    path: /opt/solr
    state: directory
    owner: solr
    group: solr
    mode: "0755"

get_url: download files

state: file: Ensure file exists; download if missing or changed
state: absent: Remove the file

- name: Download Solr archive
  get_url:
    url: https://archive.apache.org/dist/solr/9.5.0/solr-9.5.0.tgz
    dest: /tmp/solr.tgz

unarchive: extract .tar.gz or /zip

state: present: Ensure archive is extracted
state: absent: Remove extracted files
remote-src: yes: src is from remote server

- name: Extract Solr
  unarchive:
    src: /tmp/solr.tgz
    dest: /opt/
    remote_src: yes

user: manage users

state: present: Create user if missing
state: absent: Remove user
state: locked: Lock user login
state: unlocked: Unlock user login
system: yes: creates system user
create_home: yes: creates normal user
password: "{{ 'vaultPassword' | password_hash('sha512') }}"

Use ansible-value instead of hardcoding password into the password field
Setting up ansible-vault

ansible-vault create vault.yml: create the value file and open a text editor
vault.yml should go in ansible/group_vars/all/vault.yml to be accessible by all roles
Inside vault.yml, add userPassword: "MyVerySecurePassword123!"

vault.yml is encrypted on disk after

Run playbook as ansible-playbook playbook.yml --ask-vault-pass
ansible-vault view vault.yml: check the vault
ansible-vault edit vault.yml: edit the vault

- name: Create user securely
  user:
    name: username
    shell: /bin/bash
    state: present
    create_home: yes
    password: "{{ userPassword | password_hash('sha512') }}"

group: manage groups

state: present: Create user if missing
state: absent: Remove user

- name: Create solr group
  group:
    name: solr

lineinfile: modify single line in a file

state: present: Ensure line exists (insert if missing)
state: absent: Ensure line is removed
insertafter: EOF: ensures line is appended at the end

- name: Disable root SSH login
  lineinfile:
    path: /etc/ssh/sshd_config
    regexp: '^PermitRootLogin'
    line: 'PermitRootLogin no'

ufw: firewall rules

state: enabled: Enable firewall
state: disabled: Disable firewall
state: reloaded: Reload rules
state: reset: Reset firewall to default

- name: Allow SSH
  ufw:
    rule: allow
    port: 22

command: run commands

command: >: allow the command to be written on multiple lines and ansible will treat it as a single-line command
args: extra arguement for the task

chdir: /path/to/run/task: run the task in the specified directory
creates: /path: if /path already exists, then this task will not be run

- name: Check solr version
  command: solr --version

shell: run shell commands

- name: Run install script
  shell: |
    cd /tmp
    bash install_solr.sh

debug: for debug purposes: like outputting messages

msg: >-: write multi-line strings

Ansible will print each [.....] in its own line

- name: Show multiple outputs
  debug:
    msg: >-
      {{
        ['=== Zookeeper Status ==='] + zk_status.stdout_lines +
        ['"==================================================================",'] +
        ['=== Zookeeper myid: ' + (myid | string) + ' ==='] 
      }}

// Sample Output
 "=== Zookeeper Status ===",
        "● zookeeper.service - Apache ZooKeeper",
        "     Loaded: loaded ..."
        "========================================================================",
        "=== Zookeeper myid: 1 ==="

Tools