Ansible + Nuitka: One-line Installer for Smoother Employee Onboarding
The Employee Onboarding Problem
At Hexmos, we faced a significant challenge when onboarding new software engineering interns. The process of integrating them into our team and systems was very demanding. Typically, one or two senior developers would spend a significant amount of time helping interns install the many tools, packages, and extensions they needed—often numbering in the dozens.
Installing the incorrect versions of tools and packages could result in hours of debugging, which was especially challenging because we operate entirely online and the interns lack experience dealing with commands. Fixing installation issues was a nightmare.
The process was extremely time-consuming and had a negative impact on our team's overall productivity. Additionally, the need to ensure that everyone on the team was using the same versions of tools and the same shell and tool configurations (such as .bashrc, Prettier configurations) was a significant challenge. Whenever the entire team needed to use a shared tool for experimental purposes, the installation process became a laborious task that involved multiple downloads and inconveniences.
Binary Installer Idea
To tackle the issue mentioned earlier, we came up with a binary distribution idea. This involves a straightforward one-line command that downloads the binary file, grants permissions, and runs it on the intern's machine. Once executed, the binary file guides the intern's machine through the necessary onboarding tasks as per the provided instructions.
Now, the question arose: How do we create this binary file? What tools should we utilize, and how can we ensure their compatibility with various machines?
Why did we choose a binary file?
- A single executable file that does not require any external dependency
- Given interns' limited familiarity with Unix systems, a single command help to overcome the installation, and configuration issues.
- Effortless sharing via a public URL enhances accessibility.
Ansible for Installation Instructions
We opted for Ansible as our solution for creating installation instructions due to its user-friendly nature and straightforward approach. Ansible is also useful for configuration management. Meaning, we can, for example easily maintain bashrc, vscode configurations, etc through Ansible. With Ansible, we can break down each package and tool into individual tasks, allowing us to specify precise versions and configurations. This flexibility enables us to seamlessly integrate new packages into the binary by simply adding corresponding tasks to the Ansible YAML file. Another reason for packaging using Ansible is its track record. It is in active development now for more than 10+ years and is battle-tested in the system operations area.
Ansible not only reduces initial challenges but also enhances efficiency, making the onboarding experience smoother and more accessible.
Find further information and tutorials to learn more about Ansible.
Nuitka for Binary Creation
Creating a single binary file is our main goal. This file contains installation instructions and the Ansible package to execute those instructions. We chose Nuitka for this task because it provides many options and can combine Ansible and the execution file into a single, all-in-one file.
Find further information and tutorials to learn more about Nuitka.
Adapting Ansible
Creating the Playbook
First, we installed Ansible using pip: pip3 install ansible and created an Ansible playbook file. In my case, I've named it one_installer.yml. This playbook file contains instructions for installing various tools, and packages and making configurations on the intern's computers. In our playbook, we used the direct shell execution to install the packages, but it is replaceable with more advanced ansible.builtin.packages module.
Ansible API executor
Typically, Ansible playbook is executed using the command ansible-playbook one_installer.yaml. In our case, Nuitka, which we will use as the binary builder, is a Python compiler. Therefore, the Ansible playbook must be executed from a Python file. I modified an example from the Ansible documentation.
In the first 12 lines, I import the necessary Ansible modules, and then initialize the loader.
import shutil
import ansible.constants as C
from ansible.executor.task_queue_manager import TaskQueueManager
from ansible.module_utils.common.collections import ImmutableDict
from ansible.inventory.manager import InventoryManager
from ansible.parsing.dataloader import DataLoader
from ansible.vars.manager import VariableManager
from ansible import context
from ansible.executor.playbook_executor import PlaybookExecutor
import os
from ansible.utils.display import Display
import getpass
loader = DataLoader()
Configure the CLI arguments
context.CLIARGS = ImmutableDict(
listtasks=False,
listhosts=False,
syntax=False,
connection="smart",
forks=10,
become=False,
verbosity=4,
check=False,
start_at_task=None,
become_method= 'sudo',
become_user= None,
become_ask_pass= True,
)
Add the host list and sources
host_list = ["localhost"]
sources = ",".join(host_list)
In the next stage, configure the inventory, variable manager, and password collection.
inventory = InventoryManager(loader=loader, sources=sources, cache=False)
variable_manager = VariableManager(loader=loader, inventory=inventory)
sudo_password = getpass.getpass("Enter your root password :")
passwords = dict(become_pass=sudo_password)
Configure the display and task queue manager
tqm = TaskQueueManager(
inventory=inventory,
variable_manager=variable_manager,
loader=loader,
passwords=passwords,
)
display = Display()
At last set up the playbook executor and run the playbook file and clean the task queue manager on the finish
playbook_executor = PlaybookExecutor( playbooks=[os.path.join(os.path.dirname(os.path.abspath(__file__)), "one_installer.yml")],
inventory=inventory,
variable_manager=variable_manager,
loader=loader,
passwords=passwords,
)
try:
results = playbook_executor.run()
finally:
tqm.cleanup()
if loader:
loader.cleanup_all_tmp_files()
shutil.rmtree(C.DEFAULT_LOCAL_TMP, True)
The complete code is provided for reference executor.py.
Building the Binary
Package with Nuitka
We used the Nuitka Python compiler for executing the Ansible playbook in a binary. Nuitka offers two options for bundling the package.
standalone : The standalone option tells Nuitka to create a stand-alone executable file that can be run without any additional dependencies
onefile: The onefile option tells Nuitka to create a single, compressed executable file that includes all of the dependencies for your Python application.
A key point to note is that we have to manually bring in the Ansible Python package into the binary. In my case, Nuitka didn’t automatically package the entire Ansible package inside the binary. I added both the Ansible code and its data files explicitly.
To achieve this, Nuitka provides useful options like:
--include-package-data
--include-data-filesThese options let us put important data files, such as our playbook file and extra configuration files, into the binary package. This means that when we make the binary, it will have both the Ansible code and the needed data files. This ensures that the binary works as it should.
After concatenating the above options we get a command like
python3 -m nuitka --onefile \
--include-package-data=ansible:'*.py' \
--include-package-data=ansible:'*.yml' \
--include-data-files=one_installer.yml=one_installer.yml \
executor.py
Once the build is done, a binary file with the same name as the script is created in the current directory.
Local Testing
We chose to employ a Docker container for testing purposes, benefiting from its ability to provide an isolated platform and ensure package independence. This choice ensures a thorough assessment of both the Ansible installation instructions and the binary file. To facilitate this testing approach, we pulled a Ubuntu image, created a Docker container
docker pull ubuntu:20.04 \
docker run -it --name my_ubuntu_container -v ~/Docker_Share:/data ubuntu /bin/bashcopied the binary file into it, and subsequently executed it within the Docker container.
sudo cp executor.bin ~/Docker_Share
#inside the container
cd /data
./executor.binThis method of testing proves invaluable in swiftly identifying issues related to both the binary and the Ansible playbook.
Releasing Binary through Github Actions
After the local testing, we set up a GitHub workflow for creating the binary file and releasing the binary. The binary file is present in the assets part of the new releases, one of the benefits is using the same public link the recipient can download the binary directly.
Once the release is done interns can download the binary file directly using this link :
https://github.com/HexmosTech/Ansika/releases/latest/download/executor.bin
One-Line Command
This one-line command will download, give permission and execute the binary
wget -q https://github.com/HexmosTech/Ansika/releases/latest/download/executor.bin \
&& chmod +x executor.bin \
&& ./executor.bin
Build One-line Installer for Your Team/Org
You can easily integrate a one-line installer into your team's workflow to speed up the onboarding process. Start by forking the Ansika GitHub repository. After installing the necessary requirements, make modifications to the ansible-playbook ansible-playbook (one_installer.yml) file. Add tasks to install the essential software, packages, and tools your team needs. Also, provide instructions for configuring files on their machines.
To validate the implementation, you can use the Docker container method described in the "Local Testing" section. Create a Git tag and push it. GitHub actions will then automatically generate the binary and create a release for it. Finally, make adjustments as needed and share the one-line command with your employees.
Find detailed information here to build and share the one-line installer.
Conclusion
The one-line installer has significantly streamlined numerous onboarding tasks within Hexmos. Now new interns can set up packages, tools, and configurations quickly using a one-line command. I hope this article helped to give you an idea about how a binary that builds using Ansible and Nuitka solved a time and energy-consuming job.
Check out our other products such as
- Hexmos Feedback: Increase feedback frequency and specificity within your teams to boost performance.
- Lama 2: Plain-Text Powered REST API Client for Teams.
Visit the following links to learn more
Hacker News Post
