Data Sovereign Agents - Deployment
Purpose
The purpose of this document describes the different steps on how to set up a Data Sovereign(DS) Agent on an OUTSCAN platform.
Introduction
The standard Agent reports its information back to OUTSCAN, regardless where in the world that endpoint is located. If it has an Internet connection, it report back its vulnerability information. Sometimes there are regulatory requirements stipulation that all data must stay within the confines of their environment. The DS Agent resolves this by ensuring that the Agent communicates back to an Endpoint in the customers infrastructure.
DS Agent Deployment Requirement
Understanding the Requirements from DSAgent Deployment Perspective
From a high level standpoint, DSAgent deployment will look as follow.
In addition to the intercommunication between the HIAB Scheduler and the DSAgent server, both will require to be able to communicate with:
- The Object Storage deployed and used by the customer
- OUTSCAN platform hosted by OUTPOST24
The installed Agent requires the ability to communicate with the DSAgent server, meaning that this server need to be accessible from internet if any Agent is deployed outside customer premises.
DSAgent installation requirement
Do not forget that DSAgent server also require to be able to access OUTPOST24 GitHub repository to retrieve the Ansible scripts for the installation and update. We also highly recommend to register and save your customization in your own Git repository, meaning the DSAgent server will need to be able to access it as well.
Requirements
- Prior deploying the Outpost24 DS Agent, ensure that the HIAB Scheduler is updated to latest version and that it has been rebooted to ensure that services are properly restarted with new parameters from the latest version.
Outpost24 DS Agent requires a specific server with a basic CentOS 7 Linux distribution, reachable by Outpost24 HIAB and that it can reach the OUTSCAN platform to access the global PKI service and the packages repository.
Note
The server must be configured to use Etc/UTC timezone and this can be done by issuing the following command:
sudo timedatectl set-timezone UTC
BASH- Outpost24 is providing Ansible playbooks to deploy the DS Agent, so Ansible software need to be installed on a server to be able to run the Ansible playbook. The DS Agent also requires an extra CentOS 7 server to be deployed.
Ansible can be installed using the Linux distribution package manager or via the manual installation procedure described in the Ansible documentation https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html.
Note
The Ansible scripts require at least Ansible version 2.8 to run without error or adjustment due to modules that are used such as postgresql_query.
- Outpost24 highly recommend to use Git to retrieve Ansible playbooks from Outpost24 public repository.
Outpost24 highly recommend to install all software (Git, Ansible) on the DS Agent server (based on CentOS 7), and run the installation from the server to minimize issues due to Linux distribution.
Note
The Ansible script support two installation schema:
- Use of an external PostgreSQL database: This is handled by adjusting variables in the Ansible playbook.
- No use of external database: The Ansible playbook will install a PostgreSQL database on the DS Agent server.
firewalld need to be install and started.
sudo yum install firewalld sudo systemctl start firewalld sudo systemctl enable firewalld
BASHThe DS Agent require a S3-API compatible Object Storage.
Preparing the Data Sovereign Agent Server
Outpost24 recommend to create a user that can elevate privilege to become root on the DS Agent server to install Outpost24 software.
The DS Agent server requires the Ansible and Git software to be installed.
Ansible
To install Ansible run the following command:
sudo yum install -y epel-release
sudo yum install -y ansible
Git
To install Git run the following command:
sudo yum install -y git
Deploying the Data Sovereign Agent
Clone the Ansible inventory template to your localhost and change it to point at a repository on your Git server of choice.
Warning!
While the playbook and template is setup to store sensitive secrets in encrypted for in vault_vars we strongly recommend that you do not make your inventory repository publicly available.
$ git clone --recursive https://github.com/Outpost24/template-hiab.git <mydsagent>-inventory # when published $ cd <mydsagent>-inventory $ git remote set-url origin https://<mygit>.localdoman.com/<mygitaccount>/<mydsagent>.git # point to a repository on your git server and repository
BASHOnce the repository is cloned, manually adjust the configuration to match your environment.
Adjusting Ansible Configuration to Meet Your Deployment
Mandatory Inventory Configuration
The following configurations are required and must be performed.
Key Server: clavem.hosts
Add the host(s) that should run clavem service on to the group clavem. Either IP or hostnames are valid.
inventory/hosts.yml
clavem:
hosts:
<mydsagenthost>:
Agent Server: teddy_salad.hosts
Add the host(s) that should run teddy-salad service on to the group teddy_salad. Either IP or hostnames are valid.
inventory/hosts.yml
teddy_salad:
hosts:
<mydsagenthost>:
HIAB Scheduler(s) and Scanners(s): scheduler.hosts and scanner.hosts
Add the HIAB that should be used to run reports on agent targets to their respective group. Either IP or hostnames are valid.
inventory/hosts.yml
scheduler:
hosts:
<myschedulerhost>:
scanner:
hosts:
# <myscannerhost1>:
# <myscannerhost2>:
OUTSCAN platform: outscan.site_basehref and outscan.username
Add the hostname and main account username of the OUTSCAN front-end that your HIAB are enrolled to.
inventory/group_vars/all/all.yml
outscan:
site_basehref: outscan.outpost24.com
username: <myoutscanusername>
PKI Server: libellum.server
Add the hostname of the OUTSCAN PKI host.
inventory/group_vars/all/all.yml
libellum:
server: pki.outscan.outpost24.com
Database Server: teddy_salad.db.sslmode, teddy_salad.db.host, psql_ssl, and pghba_extra
Add the host running the database server for teddy-salad service.
inventory/group_vars/teddy_salad/teddy-salad.yml
teddy_salad:
db:
sslmode: disable
host: <mydsagenthost>
psql_ssl: disable
pghba_extra:
- type: host
database: teddy-salad
user: teddy-salad
address: <mydsagenthost>
authmethod: scram-sha-256
Note
The address must be in the format XXX.YYY.ZZZ.AAA/32 if /32 is not added the database will not be able to start.
Agent Server: teddy_salad.endpoints
Add the host(s) that your agent clients should call home to. Multiple entries in the list will be used as fail-over. If you want load balancing you should configure a round-robin DNS record.
inventory/group_vars/teddy_salad/teddy-salad.yml
teddy_salad:
endpoints:
- <mydsagenthost>:5046
Object Storage: teddy_salad.storage.endpoint and teddy_salad.storage.region
Add the URL of your S3 API compatible object storage and configure what region you want to use. For most S3 API other than AWS S3 the region is always us-east-1.
inventory/group_vars/teddy_salad/teddy-salad.yml
teddy_salad:
storage:
endpoint: https://s3.us-east-1.amazonaws.com
region: us-east-1
self_sign_code_signing: true
Note
In case you are using an on-premises S3 API compatible object storage such as MinIO, the configuration should be updated and you need to add an optional directive certificate_authority
as follow:
inventory/group_vars/teddy_salad/teddy-salad.yml
teddy_salad:
storage:
endpoint: https:<OBJECT_STORAGE_IP_OR_FQDN>:<OBJECT_STORAGE_PORT>
region: local
certificate_authority: /<FULL_PATH_TO_OBJECT_STORAGE_CA_FILE>
Agent Server: teddy_salad.self_sign_code_signing
The agent installers downloaded are created on the fly to embed your configuration. This means they cannot be signed a head of time. Therefore, a code signing certificate needs to be provided. If you want to generate a self signed you can enable this setting.
inventory/group_vars/teddy_salad/teddy-salad.yml
teddy_salad:
self_sign_code_signing: true
Optional Inventory configuration
Key Server: clavem.ensure_commit.check and clavem.ensure_commit.fail
By default the playbook stops and prompt you to commit certificates and keys that are to be shared across multiple hosts to your inventory, failure to do so will abort the playbook. The following settings allows you to suppress the behavior, either the checking or just the failure.
inventory/group_vars/clavem/clavem.yml
clavem:
ensure_commit:
check: false
fail: false
Agent server: teddy_salad.ensure_commit.check and teddy_salad.ensure_commit.fail
By default the playbook stops and prompt you to commit certificates and keys that are to be shared across multiple hosts to your inventory, failure to do so will abort the playbook. The following settings will allow you to suppress the behavior, either the checking or just the failure.
inventory/group_vars/teddy_salad/teddy-salad.yml
teddy_salad:
ensure_commit:
check: false
fail: false
Mandatory Ansible Vault Configuration
Note
The following configurations are required and must be performed.
Replace the default passphrase ('passphrase') of the vault and then edit it.
$ ansible-vault rekey vault_vars/hiab.yml
Vault password: passphrase
New Vault password: <myvaultpassphrase>
Confirm New Vault password: <myvaultpassphrase>
Rekey successful
$ ansible-vault edit vault_vars/hiab.yml
Object storage credentials: vault_teddy_salad_storage_id and vault_teddy_salad_storage_key
Provide a access and secret key to the configured object storage.
vault_vars/hiab.yml
vault_teddy_salad_storage_id: <myaccesskey>
vault_teddy_salad_storage_key: <mysecretkey>
Database Server Password: vault_teddy_salad_db_password
Provide a database password.
vault_vars/hiab.yml
vault_teddy_salad_db_password: <mydatabasepassword>
OUTSCAN Account Password: vault_outscan_password
Provide the password for your OUTSCAN main user account.
vault_vars/hiab.yml
vault_outscan_password: <myoutscanaccountpassword>
Key Server Passphrase: vault_clavem_encryption_key_passphrase
Provide a passphrase to be used for private keys used for encrypting data in object storage.
vault_vars/hiab.yml
vault_clavem_encryption_key_passphrase: <myencryptionkeypassphrase>
Agent Server Passphrase: vault_codesigning_passphrase
Provide a passphrase to be used for the private key used for signing the installers.
vault_vars/hiab.yml
vault_codesigning_passphrase: <mycodesigningkeypassphrase>
Running Ansible Scripts in the Environment
To configure the DS Agent software, run the Ansible playbook as follow:
You will be prompted for your sudo password and vault passphrase in the beginning of the run and by default at two points during the run you might be prompted to commit certificates and private keys to your repository.
$ ansible-playbook -KD --ask-vault-pass hiab-ansible/playbooks/hiab.yml
BECOME password: <mysudopassword>
Vault password: <myvaultpassphrase>
PLAY [Install clavem-server] *****************************************************************************************
TASK [Gathering Facts] ***********************************************************************************************
ok: [mydsagenthost]
...
TASK [ensure_commit : Confirm committal of key and cert] *************************************************************
[ensure_commit : Confirm committal of key and cert]
A files/etc/pki/tls/certs/clavem/encryption.crt
A files/etc/pki/tls/certs/clavem/encryption.crt.csr
A files/etc/pki/tls/private/clavem/encryption.key
Make sure you have committed the listed files:
ok: [mydsagenthost]
TASK [ensure_commit : Fail if uncommited keys and certs] *************************************************************
changed: [mydsagenthost]
...
TASK [ensure_commit : Confirm committal of key and cert] *************************************************************
[ensure_commit : Confirm committal of key and cert]
A files/etc/pki/tls/certs/teddy-salad/codesign.crt
A files/etc/pki/tls/private/teddy-salad/codesign.key
Make sure you have committed the listed files:
ok: [mydsagenthost]
TASK [ensure_commit : Fail if uncommited keys and certs] *************************************************************
changed: [mydsagenthost]
...
PLAY RECAP ***********************************************************************************************************
mydsagenthost : ok=96 changed=40 unreachable=0 failed=0 skipped=66 rescued=0 ignored=0
myschedulerhost : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
When the command returns without any failed or unreachable tasks the host(s) are configured and ready to be used by your HIAB deployment.
Setting Up an Data Sovereign Agent Using System Proxy
Linux
Update the config files as below:
// /etc/yum.conf proxy=http://<PROXY_IP>:<PROXY_PORT>
// /etc/profile export HTTP_PROXY="http://<PROXY_IP>:<PROXY_PORT>" export HTTPS_PROXY="http://<PROXY_IP>:<PROXY_PORT>" export NO_PROXY="localhost,127.0.0.1,<HIAB_SCHEDULER_IP>,<OBJECT_STORAGE_IP>"
// /etc/sysconfig/teddy-salad HTTP_PROXY="http://<PROXY_IP>:<PROXY_PORT>" HTTPS_PROXY="http://<PROXY_IP>:<PROXY_PORT>" NO_PROXY="localhost,127.0.0.1,<HIAB_SCHEDULER_IP>,<OBJECT_STORAGE_IP>"
Download and install o24-agent.
Note
Any changes to the agent integration requires a restart of the HIAB for the changes to take full effect.Configuring Data Sovereign Agents on a HIAB Scheduler Portal
See Configure Data Sovereign Agents on HIAB Scheduler Portal document on how to configure a Data Sovereign Agent on a HIAB scheduler.
References
- Ansible documentation - https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html
Copyright
© 2023 Outpost24® All rights reserved. This document may only be redistributed unedited and unaltered. This document may be cited and referenced only if clearly crediting Outpost24® and this document as the source. Any other reproduction and redistribution in print or electronically is strictly prohibited without explicit permission.
Trademark
Outpost24® and OUTSCAN™ are trademarks of Outpost24® and its affiliated companies. All other brand names, product names or trademarks belong to their respective owners.