minio

0

Configure S3 object storage as snapshot repository for elasticsearch on ECK

Posted on by jlim0930

Creating S3 repository for elasticsearch using minio as the s3 object storage backend Please note that this should not be used in production, this is a lab environment to provide guidance for configuration. With many kubernetes environments you can have the cloud provided object storage that can be configured with ECK and used as your snapshot repositories, this guide is more for 3rd party object storage that you might have if you are not on cloud and for modifying existing deployments to configure new snapshot repositories. I will perform the steps in GKE environment onto the default namespace Install minio… Continue Reading →

0

Create secrets to use with your elastic stack to create keystore in kubernetes – minio example – helm example included

Posted on by jlim0930

For this example I will stand up a very simple minio server on my localhost. Create kubernetes secrets for the s3.client.default.access_key and s3.client.default.secret_key. Configure my elasticsearch pod with initContainer to install the repository-s3 plugin and secureSettings to create the keystore. minio server This is a very simple, not secure setup just for testing $ mkdir data $ wget https://dl.min.io/server/minio/release/linux-amd64/minio $ chmod +x minio $ ./minio server ./data API: http://192.168.1.251:9000 http://172.17.0.1:9000 http://192.168.122.1:9000 http://192.168.49.1:9000 http://127.0.0.1:9000 RootUser: minioadmin RootPass: minioadmin Console: http://192.168.1.251:36012 http://172.17.0.1:36012 http://192.168.122.1:36012 http://192.168.49.1:36012 http://127.0.0.1:36012 RootUser: minioadmin RootPass: minioadmin Command-line: https://docs.min.io/docs/minio-client-quickstart-guide $ mc alias set myminio http://192.168.1.251:9000 minioadmin minioadmin Instead of getting… Continue Reading →

1

ECE(elastic cloud enterprise) snapshot repository using object storage(minio) with self-signed TLS

Posted on by jlim0930

Just FYI this is a complete rewrite of the previous article. It should be better organized and more closely align with real world situations. We will stand up a minio server with self-signed or internally signed SSL certificate that is not publicly trusted. This guide will work with any s3 compliant object storage not just minio. The certificate that we will create will be for internal use but you can and should create a more secure certificate to be used in your environment. If you have s3 compliant object storage with publicly trusted signed SSL certificate then all you would… Continue Reading →

1

ECE(Elasticsearch Cloud Enterprise) snapshots with minio – TLS!

Posted on by jlim0930

PLEASE use THIS POST instead, as many items were updated and changed. Configuring snapshots for ECE(Elasticsearch Cloud Enterprise) or deployments for ECE from a S3 object storage or on a s3 compliant storage with publicly trusted certificates are easy but how do you configure the snapshot repository if you are using self-signed or internally signed certificates ? There are 2 ways of configuring snapshot repository and snapshots on ECE. First you can configure the snapshot repository on ECE. Once you configure the snapshot repository in the Admin UI, you can configure found-snapshots for each deployment via the Admin UI. Unfortunately,… Continue Reading →

2

Elastic Cloud on kubernetes (ECK) on minikube

Posted on by jlim0930

ECK is Elastic cloud on kubernetes – Kubernetes Operator pattern that extends basic kubernetes orchestration to easily deploy, secure, upgrade Elasticsearch and the rest of the stack such as kibana, logstash, various beats, and much more. In my previous article I wrote about my kube.sh script which is a wrapper that installs a minikube environment to run your kubernetes projects. We will use the wrapper to deploy our kubernetes environment and start deploying elasticsearch using ECK. The steps below are not limited to my minikube wrapper script but you can follow the same process for any kubernetes environment. Lets get… Continue Reading →

0

quick, fast & dirty minio server using docker

Posted on by jlim0930

I need a quick & fast method of standing up a not so fancy minio server to test some s3 functionality. Nothing fancy was needed nor the need for multiple nodes or large datastores, but I did need it something that is easily installed/cleanedup and repeated. minio.sh was born! Features tested on macOS, linux, wsl creates myminio directory in your ${HOME} cleanup cleans off everything do not need root just need to be part of the docker group Requirements docker is needed. any kind of s3 client like mc, s3cmd is good to have Installation $ curl -fsSL https://raw.githubusercontent.com/jlim0930/scripts/master/minio.sh -o… Continue Reading →

6

Quickly deploy elasticsearch with docker

Posted on by jlim0930

UPDATE: there is a new version that was released in March 2022 and you can read about it here I needed a way to quickly stand up various versions of elasticsearch for testing. Sometimes just 1 instance was needed and at other time a small cluster and configure it with TLS and security settings. I spent way too much time installing and reinstalling the software on my vm and knew that this will not work well for me. So I set out to look for a light weight, portable, fast, re-produce-able solution that I can quickly stand up and delete… Continue Reading →