Elasticsearch service configurations

TABLE OF CONTENTS

Introduction

In case, the Elasticsearch service configured is throwing out of memory exception and the Elasticsearch service stops unexpectedly, it may be caused because the JVM Max heap memory configured for the elasticsearch may be reached out. By default, the JVM heap size is 1GB.

For Example: Suppose, in an Elasticsearch service, the JVM Heap max size was set to 1 GB, and the catalog data size is too high, the chunk size used to fetch records from SQL database tables to insert into elastic search index (i.e PublishProductFetchChunkSize) was set to 10000.

Then there might be the case where not enough space is available in the JVM heap to hold the chunk data required for creating an index. As a result, the Elasticsearch service might throw an out-of-memory exception and the service may be stopped unexpectedly. 

To overcome this issue, some configurations need to be modified mentioned below.

API Web.config Key Configuration

In the API Web.config there is a “PublishProductFetchChunkSize” key, this key is used to fetch the number of records that need to be published on Elasticsearch.

In this case, Znode recommends to reduce the “PublishProductFetchChunkSize” key value.

Suppose, the “PublishProductFetchChunkSize” key value is set to 15000, then it can be reduced to 5000. If the key value is empty or zero return the default chunk size i.e. 10000. By reducing this Key size, we can reduce the burden that is causing the JVM heap size. This Key value may vary depending on server/system configuration & space capability.

Elasticsearch service configuration

You can configure the Elasticsearch Properties by following the below steps:

  • Go to the folder structure where the Elasticsearch service is configured.

  • Open the bin folder available in the installed elasticsearch folder 

Ex: C:\Program Files\Elastic\Elasticsearch\7.16.2\bin

  • Make sure Elasticsearch-service.bat and elasticsearch-service-mgr.exe files are available in the folder. In case, these files are not there then try using the way mentioned in the note.

  • Open CMD in administrative mode.

  • [path till bin folder available in the installed elasticsearch folder]>elasticsearch-service.bat manager

It will open Elasticsearch Properties as shown below:

  • Now go to the 'Java' tab and change settings based on your requirement. In case, the Java tab is not there then try using the way mentioned in the note.

Example:



For Elasticsearch clusters, ECE gives 50% of the available memory to the JVM heap used by Elasticsearch, while leaving the other 50% for the operating system.

For More Details refer:

JVM Heap Sizes | Elastic Cloud Enterprise Reference [2.0].

Thread stack size can also be tune with the Initial memory pool, and Maximum memory pool. 

  • After your change, you need to restart the service.

Note - 

If the elasticsearch.bat file and elasticsearch-service-mgr.exe files are not available in the bin folder then the same setting can be configured using ES_JAVA_OPTS environment variable:

  • Add an environment variable under System variables with Variable name: ES_JAVA_OPTS and Variable value: -Xms2g -Xmx2g 

  • Here, -Xms2g allocate at minimum 2GB of heap and -Xmx2g allocate at maximum 2GB of heap.

  • To set in MB, with Variable name: ES_JAVA_OPTS and Variable value: -Xms256m -Xmx512m

  • Here, -Xms256m allocate at minimum 256MBs of heap and -Xmx512m allocate at maximum 512MBs of heap.

  • Xms represents the initial size of total heap space

  • Xmx represents the maximum size of total heap space

References:

https://www.elastic.co/guide/en/elasticsearch/guide/current/indexing-performance.html#_using_and_sizing_bulk_requests


Did you find it helpful? Yes No

Send feedback
Sorry we couldn't be helpful. Help us improve this article with your feedback.