elasticsearch

2

Updated deploy-elastic script – version 8

Posted on by jlim0930

Last update to version 7 is listed here The original post for the deploy-elastic.sh script is here. Changes: fully tested from 6.x-8.2(will most likely work with future 8.x versions) decoupled non stack modes from checking versions so that you can add the secondary components like monitoring/fleet/apm/enterprise-search using different version than the stack version. Just install the stack first then install the secondary components with different versions. Secondary versions can not be higher than the stack version. This will allow you to test using different versions fleet setting is now auto-populated. The script will gather the machines external IP and set… Continue Reading →

0

Elasticsearch on k8s (ECK) All about passwords

Posted on by jlim0930

Lets talk about passwords! How is it set, where is it, how do you reset it, how do you force it? When you do a simple deployment such as apiVersion: elasticsearch.k8s.elastic.co/v1 kind: Elasticsearch metadata: name: quickstart spec: version: 8.1.1 nodeSets: – name: default count: 1 config: node.store.allow_mmap: false Where is your password stored and how do you access it? For this article we will assume that your deployment name is quickstart Default The elastic users password is stored in secrets along with all the other secrets. It usually takes form of <DEPLYMENT NAME>-es-elastic-user and to get the password you can… Continue Reading →

9

Updated deploy-elastic script – version 7

Posted on by jlim0930

UPDATE 3/30/2022 – another update was made to make fleet deployments easier. This is explained on this post The original post for the deploy-elastic.sh script is here. Made some updates so I thought I would post it on a new post. Changes: 8.x is now supported. You can use the script for any versions 6.x-8.x. full – modes are removed. The script is additive so if you deploy only the stack and want to add features on top you can run it again with the feature you want to add. You can add multiple features to your stack. Starting stack… Continue Reading →

0

Create secrets to use with your elastic stack to create keystore in kubernetes – minio example – helm example included

Posted on by jlim0930

For this example I will stand up a very simple minio server on my localhost. Create kubernetes secrets for the s3.client.default.access_key and s3.client.default.secret_key. Configure my elasticsearch pod with initContainer to install the repository-s3 plugin and secureSettings to create the keystore. minio server This is a very simple, not secure setup just for testing $ mkdir data $ wget https://dl.min.io/server/minio/release/linux-amd64/minio $ chmod +x minio $ ./minio server ./data API: http://192.168.1.251:9000 http://172.17.0.1:9000 http://192.168.122.1:9000 http://192.168.49.1:9000 http://127.0.0.1:9000 RootUser: minioadmin RootPass: minioadmin Console: http://192.168.1.251:36012 http://172.17.0.1:36012 http://192.168.122.1:36012 http://192.168.49.1:36012 http://127.0.0.1:36012 RootUser: minioadmin RootPass: minioadmin Command-line: https://docs.min.io/docs/minio-client-quickstart-guide $ mc alias set myminio http://192.168.1.251:9000 minioadmin minioadmin Instead of getting… Continue Reading →

2

elasticsearch stack monitoring on kubernetes(ECK)

Posted on by jlim0930

UPDATE: Since ECK operator 1.7 there is a new way to deploy stack monitoring. I’ve added the section at the end to cover for this. The old method still works and depending on your use case you can still use the old method versus new. The main difference between the OLD and NEW is that the OLD method uses filebeat and metricbeat pods while the new method uses filebeat and metricbeat sidecar containers. Lets get started with monitoring your elastic stack in kubernetes. There are multiple ways of doing this and I will divide this up into 3 sections, using… Continue Reading →

1

elasticsearch 8.x configure roles and users for stack monitoring

Posted on by jlim0930

A lot of changes were made with elasticsearch 8.0 release but the main one that affected me the most was the ability to use the default elastic(superuser role) to access system indices. According to https://www.elastic.co/guide/en/elasticsearch/reference/current/release-highlights.html#_better_protection_for_system_indices we no longer have write access to system indices with the superuser role. When configuring stack monitoring using metricbeats this is an issue if you’ve been just using the elastic user to configure the monitoring. You can add allow_restricted_indices to the role to enable it back however its the lazy and insecure method. I’ve summarized the creation of roles and users for metricbeats and filebeats… Continue Reading →

0

How to run setup for various beats(filebeat, metricbeat, & more) in kubernetes to load dashboards and more

Posted on by jlim0930

Whenever you install beats or update beats its best if you run the setup so that the setup will update your index templates, ILM, visualizations, etc. I found that its always best before installing or upgrading to use a seed host to just run the setup before deploying or updating across your environment and to turn off template updates and dashboard loading on the beats locally so that your elasticsearch cluster is not flooded. In a baremetal environment this is easy to do but in docker and in kubernetes it gets a bit difficult. Sure you can setup a initContainer… Continue Reading →

1

ECE(elastic cloud enterprise) snapshot repository using object storage(minio) with self-signed TLS

Posted on by jlim0930

Just FYI this is a complete rewrite of the previous article. It should be better organized and more closely align with real world situations. We will stand up a minio server with self-signed or internally signed SSL certificate that is not publicly trusted. This guide will work with any s3 compliant object storage not just minio. The certificate that we will create will be for internal use but you can and should create a more secure certificate to be used in your environment. If you have s3 compliant object storage with publicly trusted signed SSL certificate then all you would… Continue Reading →

0

Generate test data using elastic’s makelogs

Posted on by jlim0930

Updated 5/11/2023 When I am testing for various issues I find that I need to create some test data. The built in sample data is great but sometimes you need to keep ingesting data or ingest some older data to reproduce issues. I found this repo and it was the answers to my issue. I did not want to install this and wanted to containerize it so that I can leave my system clean yet be able to use this and re-use it again. makelogs runs fine as it is however if your ES endpoint is secured with TLS and… Continue Reading →

1

ECE(Elasticsearch Cloud Enterprise) snapshots with minio – TLS!

Posted on by jlim0930

PLEASE use THIS POST instead, as many items were updated and changed. Configuring snapshots for ECE(Elasticsearch Cloud Enterprise) or deployments for ECE from a S3 object storage or on a s3 compliant storage with publicly trusted certificates are easy but how do you configure the snapshot repository if you are using self-signed or internally signed certificates ? There are 2 ways of configuring snapshot repository and snapshots on ECE. First you can configure the snapshot repository on ECE. Once you configure the snapshot repository in the Admin UI, you can configure found-snapshots for each deployment via the Admin UI. Unfortunately,… Continue Reading →