I’ve created 2 Ansible Roles (chilcano.apache-nifi and chilcano.apache-nifi-toolkit) to automate the creation of a multi-node and secure NiFi cluster. The chilcano.apache-nifi Ansible Role doesn’t implement Cluster State coordination through Apache ZooKeeper, and the TLS Toolkit Standalone mode has been implemented by the chilcano.apache-nifi-toolkit Ansible Role.
The purpose of this blog post is explaining how to automate the creation of 5 instances of Apache NiFi, secure and not secure.
The first NiFi instance
nf1 will be a standalone instance running over HTTP. The second instance will will be a customized instance running over HTTPS with Client Certificate authentication. The third, fourth and fifth instances will run over HTTPS with Client Certificate authentication with configuration provided for NiFi TLS Toolkit. The configuration (key-pair, Java key stores and certificates) will be generated in other VM instance provided for
chilcano.apache-nifi-toolkit Ansible Role.
I’ve created an Ansible Playbook for you, you can download from this Git repository.
How to use it - steps
1. Clone the Ansible playbooks
$ git clone https://github.com/chilcano/ansible-apache-nifi-multi-nodes
chilcano.apache-nifi-toolkit Ansible Roles
$ cd ansible-apache-nifi-multi-nodes $ ansible-galaxy install -r playbooks/requirements.yml
3. Create all VMs with Vagrant
Create all 6 VMs by using Vagrant.
$ cd infra/Vagrant $ vagrant up
4. Ansible provisioning with Vagrant
Now, I’m going to provision (run Ansible Playbooks) through Vagrant. This step will install Apache NiFi TLS Toolkit in the
nftk1 VM, once provisioned, Vagrant will provision 5 VMs following the Ansible Playbook
playbooks/main.yml. It is very important to start the provision of all NiFi instances after provisioning
nftk1. In the
playbooks/main.yml you will see the
nftk1 is declared on top and after
nf1, nf2, nf3, nf4 and
$ vagrant provision $ vagrant status Current machine states: nftk1 running (virtualbox) nf5 running (virtualbox) nf4 running (virtualbox) nf3 running (virtualbox) nf2 running (virtualbox) nf1 running (virtualbox) This environment represents multiple VMs. The VMs are all listed above with their current state. For more information about a specific VM, run `vagrant status NAME`.
Now we can verify if all instances are running as expected, before we have to install the
Client Certificate ( CN=chilcano_OU=INTIX.p12 ) generated in our browser.
Client Certificate only is required when connecting to
nf2, nf3, nf4 and
nf5 because these instances are running over HTTPS with Mutual SSL/TLS Authentication (based on Client Certificate).
Open the URL (
http://nf5:9443/nifi) from Firefox. Instead of hostname you can use the IP address (see
- Improve the Ansible Role
chilcano.apache-nifito implement Cluster Status coordination through
- Improve the Ansible Role
chilcano.apache-nifi-toolkitto implement Client/Server mode.
- Deploy a sample DataFlow in NiFi.