This post was originally published on this site

A Customer has extended his Oracle Database Appliance X6-L with Oracle Database Appliance Memory Expansion Kit from 192GB to 384GB. After the first startup, the odacli tool has recognized the additional available memory and gave us the suggested value for the new HugePage settings.

The ODA has the newest available patchset applied, OS is Oracle Enterprise Linux 6.10.

Update the OS Configuration

Oracle suggested the new values for the HugePage settings to 135350MB. 

[[email protected] ~]# odacli list-osconfigurations 

Parameter User ConfiguredValue SuggestedValue 
--------------- --------------- ------------------------------ --------------- 
Memlock grid 386716274KB 386716168KB 
Memlock oracle 386716274KB 386716168KB 
HugePage default 67815MB 135350MB

According the Oracle documentation Oracle® Database Appliance X6-2S/X6-2M/X6-2L Deployment and User’s Guide – Section 13.14 – we updated the OS configuration by odacli. The job was finished after some seconds.

odacli update-osconfigurations (page 13-70)
Use the command odacli update-osconfigurations to update the HugePage and memlock values.

[[email protected]]# odacli update-osconfigurations 

Verfication

The HugePage Settings were updated now.

[[email protected] ~]# odacli list-osconfigurations 

Parameter User ConfiguredValue SuggestedValue 
--------------- --------------- ------------------------------ --------------- 
Memlock grid 386716168KB 386716168KB 
Memlock oracle 386716168KB 386716168KB 
HugePage default 135350MB 135350MB

Time for a hardware restart.

Old HugePage Values after ODA Restart

But, after the machine has started, the old HugePage values were back again. But who has overwritten the settings?

[[email protected] ~]# odacli list-osconfigurations 

Parameter User ConfiguredValue SuggestedValue 
--------------- --------------- ------------------------------ --------------- 
Memlock grid 386716274KB 386716168KB 
Memlock oracle 386716274KB 386716168KB 
HugePage default 67815MB 135350MB

An SR has solved the problem, the are two files in the OS which are responsible for that behavior:

  • /etc/sysctl.d/99-initial-sysctl.conf
  • /etc/sysctl.d/99-oracle-rdbms-server-12cR1-preinstall-sysctl.conf

We take a look at the values:

[[email protected] ~]# cat /etc/sysctl.d/99-initial-sysctl.conf | grep huge
vm.nr_hugepages=67815

[[email protected] ~]# cat /etc/sysctl.d/99-oracle-rdbms-server-12cR1-preinstall-sysctl.conf | grep huge
vm.nr_hugepages=67815

Solution: Remove these two Files

First, we updated the OS Configuration again.

[[email protected] ~]# odacli update-osconfigurations

Second, we removed the files.

[[email protected] ~]# rm /etc/sysctl.d/99-initial-sysctl.conf
[[email protected] ~]# rm /etc/sysctl.d/99-oracle-rdbms-server-12cR1-preinstall-sysctl.conf

Third, we restarted the Oracle Database Appliance.

Finally

HugePage settings are persistent now. By the way, there is an My Oracle Support note for the OS Level 6.10 which describes such a case too:

Changes in sysctl.conf File are not Persistent after Reboot (Doc ID 2477376.1)