Payara-Server and RAM consumption in dockerized environment

Technical Discussion
1

Hello,

I use docker to manage our payara cluster. It is composed of test instances and 4 production instances that I create as follows:

[...]
    ${PAYARA_DIR}/bin/asadmin --user=${ADMIN_USER} --passwordfile=${PASSWORD_FILE} --port=4848 create-instance --node node-production --config instance-production --portbase=21000 instance-production-1                       && \
    ${PAYARA_DIR}/bin/asadmin --user=${ADMIN_USER} --passwordfile=${PASSWORD_FILE} --port=4848 create-instance --node node-production --config instance-production --portbase=22000 instance-production-2                       && \
    ${PAYARA_DIR}/bin/asadmin --user=${ADMIN_USER} --passwordfile=${PASSWORD_FILE} --port=4848 create-instance --node node-production --config instance-production --portbase=23000 instance-production-3                       && \
    ${PAYARA_DIR}/bin/asadmin --user=${ADMIN_USER} --passwordfile=${PASSWORD_FILE} --port=4848 create-instance --node node-production --config instance-production --portbase=24000 instance-production-4                       && \
    ${PAYARA_DIR}/bin/asadmin --user=${ADMIN_USER} --passwordfile=${PASSWORD_FILE} --port=4848 create-instance --node node-recette    --config instance-recette    --portbase=31000 instance-recette-1                          && \
    ${PAYARA_DIR}/bin/asadmin --user=${ADMIN_USER} --passwordfile=${PASSWORD_FILE} --port=4848 create-instance --node node-recette    --config instance-recette    --portbase=32000 instance-recette-2                          && \
[...]

The current memory usage is as follows, which does not leave much free memory. The IT department refuses to increase the RAM for this VM.

PageSize:4KB   RAM-Memory  Swap-Space       High-Memory     Low-Memory
Total (MB)        15885.3      1024.0       - not in use   - not in use
Free  (MB)          897.7      1022.7
Free Percent         5.7%       99.9%

Are there any commands I can use when generating my image to limit the memory impact of my instances? For example, as I have seven processes (4 prod, 2 test, and 1 payara-admin) allocate 10% to each process? Or limit to 80% for all 7 payara processes?

Thanks for your help
Thierry

2

Hi,

I guess we use the nmon to monitor the memory usage. It is a summarized/overall which may not enough to drill down who consumes our resources.

Anyhow there is another table under your captured picture as the following example:-

Note:

  • With the clean re-started VM without any service except the OS it self.
  • I have 8 GBs of RAM

001-nmon
001-nmon733×197 73.8 KB

That table informs us about the buffer and cached.

If we use the free command it may display as the following: -

001-free
001-free705×108 26.4 KB

We may see that even there is no service started, the OS reserves the buffer and cached 1.4G and causes the free to be 5.7G

This may be one of the reason that we have low free memory in overall.

If we know how many buffer and cached used by the OS and feel that each instance consume more memory, we may drill down and find out further.

3

Hello @charlee_ch
Thank you for your answer

I do use nmon to monitor these metrics.
Here are several captures below:

  • the first one just after rebooting the VM and before starting the payara container
  • the second one just after the start of the payara container
  • the third one, almost 3 hours after the start of the payara container

I also displayed on the last capture (I didn’t think about the first ones) cpu and processes

nmon-os-just-started

nmon-payara-just-started

nmon-payara-after-3-hours
nmon-payara-after-3-hours955×586 43.1 KB

4

metrics state after 6 hours running Payara :

nmon-payara-after-6-hours
nmon-payara-after-6-hours952×617 45.8 KB

5

The buffer and cached is around 3+ GB and free is around 2 GB which mean that we have around 5 GB available memory out of 16 GB overall.

Furthermore, for each instance domain.xml we may limit the heap size as something like the following (or less for testing purpose): -

<jvm-options>-Xms512m</jvm-options>
<jvm-options>-Xmx1024m</jvm-options>

Anyhow the proper value is based on our application requirement which may be some result from the load testing or the monitoring in the production.