0

Mounting certificates/CAs for elasticsearch pods in k8s for custom configurations – ECK

Posted on by jlim0930

When you deploy elasticsearch in k8s via ECK by default it takes care of the transport certificates/http certificates/CA creation if you don’t use your own custom ones. Lets say that your cluster is up and running and now you are trying to configure something like SAML/LDAP/OIDC or something else that requires content to be made available to the elasticsearch pod but its a bit sensitive so you need it to be stored as a secret such as CA, certificates, etc. How would you go about doing this? Following example will add a custom certificate authority to be used with your… Continue Reading →

0

Updated deploy-elastic script – version 9

Posted on by jlim0930

Last update to version 8 is listed here The original post for the deploy-elastic.sh script is here Changes: Starting 7.12.x added node.roles. es01 & es02 is tagged as data_content & data_hot and es03 is tagged as data_frozen. This will allow users to play with the frozen tier, configure searchable snapshots, partically mounted indices, restored indices, and more. (make sure to run the script with snapshot option so that minio is stood up) LDAP mode was added. To test various role_mappings and for basic ldap authentication configurations and settings LDAP mode was added and it will work with any versions of… Continue Reading →

1

Elastic Fleet Server & elastic-agent common troubleshooting

Posted on by jlim0930

Common troubleshooting items for any fleet & elastic-agent issues can be found on https://www.elastic.co/guide/en/fleet/current/fleet-troubleshooting.html however there are a few items that I would like to add. Whitespaces When you start the elastic-agent on an endpoint and even if you have everything configured for your elastic-agent, the right policy, integrations, etc however no data is going into elasticsearch. The most common issue is that there is something wrong with your fleet settings. This can be edited from kibana -> fleet -> settings. Either your ES endpoint / Fleet server endpoint / Advanced YAML settings are wrong or there might even be… Continue Reading →

4

Fleet server with logstash output – elastic-agent

Posted on by jlim0930

I think one of the biggest issue with fleet and elastic-agent was that it was limited on the outputs so if you had tons and tons of elastic-agents in the wild it would all connect back to your elasticsearch and can overwhelm the cluster. Starting 8.2 of elasticsearch logstash output type was introduced.. it is still in BETA at this time but I just tried it and it worked very nicely! The steps to setup the logstash output is listed on https://www.elastic.co/guide/en/fleet/8.2/secure-logstash-connections.html but I wanted to try it out and document it. Install & configure logstash Install yum install logstash-8.2.2… Continue Reading →

1

Google Cloud scripts

Posted on by jlim0930

At work we use google cloud as our lab to stand up and take down instances. As I work, I noticed that I usually stand up the same type of instance and run the same post install all the time so instead of doing this manually I decided to create a script for the compute instance and another for the kubernetes cluster. Compute Instance gcp.sh – script to automate creating and deletion of compute instance script is located: https://github.com/jlim0930/scripts/blob/master/gcp.sh edit gcp_name & gcp_project to match your environment. Also zone and region as needed – The script will stand up a… Continue Reading →

2

Updated deploy-elastic script – version 8

Posted on by jlim0930

Last update to version 7 is listed here The original post for the deploy-elastic.sh script is here. Changes: fully tested from 6.x-8.2(will most likely work with future 8.x versions) decoupled non stack modes from checking versions so that you can add the secondary components like monitoring/fleet/apm/enterprise-search using different version than the stack version. Just install the stack first then install the secondary components with different versions. Secondary versions can not be higher than the stack version. This will allow you to test using different versions fleet setting is now auto-populated. The script will gather the machines external IP and set… Continue Reading →

0

Elasticsearch on k8s (ECK) All about passwords

Posted on by jlim0930

Lets talk about passwords! How is it set, where is it, how do you reset it, how do you force it? When you do a simple deployment such as apiVersion: elasticsearch.k8s.elastic.co/v1 kind: Elasticsearch metadata: name: quickstart spec: version: 8.1.1 nodeSets: – name: default count: 1 config: node.store.allow_mmap: false Where is your password stored and how do you access it? For this article we will assume that your deployment name is quickstart Default The elastic users password is stored in secrets along with all the other secrets. It usually takes form of <DEPLYMENT NAME>-es-elastic-user and to get the password you can… Continue Reading →

9

Updated deploy-elastic script – version 7

Posted on by jlim0930

UPDATE 3/30/2022 – another update was made to make fleet deployments easier. This is explained on this post The original post for the deploy-elastic.sh script is here. Made some updates so I thought I would post it on a new post. Changes: 8.x is now supported. You can use the script for any versions 6.x-8.x. full – modes are removed. The script is additive so if you deploy only the stack and want to add features on top you can run it again with the feature you want to add. You can add multiple features to your stack. Starting stack… Continue Reading →

0

Create secrets to use with your elastic stack to create keystore in kubernetes – minio example – helm example included

Posted on by jlim0930

For this example I will stand up a very simple minio server on my localhost. Create kubernetes secrets for the s3.client.default.access_key and s3.client.default.secret_key. Configure my elasticsearch pod with initContainer to install the repository-s3 plugin and secureSettings to create the keystore. minio server This is a very simple, not secure setup just for testing $ mkdir data $ wget https://dl.min.io/server/minio/release/linux-amd64/minio $ chmod +x minio $ ./minio server ./data API: http://192.168.1.251:9000 http://172.17.0.1:9000 http://192.168.122.1:9000 http://192.168.49.1:9000 http://127.0.0.1:9000 RootUser: minioadmin RootPass: minioadmin Console: http://192.168.1.251:36012 http://172.17.0.1:36012 http://192.168.122.1:36012 http://192.168.49.1:36012 http://127.0.0.1:36012 RootUser: minioadmin RootPass: minioadmin Command-line: https://docs.min.io/docs/minio-client-quickstart-guide $ mc alias set myminio http://192.168.1.251:9000 minioadmin minioadmin Instead of getting… Continue Reading →