
         
          jklech@redhat.com

Sujata Kurup

Red Hat Customer Content Services


         
          skurup@redhat.com

Khushbu Borole

Red Hat Customer Content Services


         
          kborole@redhat.com

Marie Dolezelova

Red Hat Customer Content Services

Mark Flitter

Red Hat Customer Content Services

Douglas Silas

Red Hat Customer Content Services

Eliska Slobodova

Red Hat Customer Content Services

Jaromir Hradilek

Red Hat Customer Content Services

Maxim Svistunov

Red Hat Customer Content Services

Robert Krátký

Red Hat Customer Content Services

Stephen Wadeley

Red Hat Customer Content Services

Florian Nadge

Red Hat Customer Content Services

Legal Notice

Abstract

The Kernel Administration Guide documents tasks for maintaining the Red Hat Enterprise Linux 7 kernel. This release, includes information on using kpatch, managing kernel modules, and manually updating the kernel.

Preface

The Kernel Administration Guide describes working with the kernel and shows several practical tasks. Beginning with information on using kernel modules, the guide then covers interaction with the sysfs facility, manual upgrade of the kernel and using kpatch. The guide also introduces the crash dump mechanism, which steps through the process of setting up and testing vmcore collection in the event of a kernel failure. The Kernel Administration Guide also covers selected use cases of managing the kernel and includes reference material about command line options, kernel tunables (also known as switches), and a brief discussion of kernel features.

Chapter 1. Working with kernel modules

This Chapter explains: What is a kernel module. How to use the kmod utilities to manage modules and their dependencies. How to configure module parameters to control behavior of the kernel modules. How to load modules at boot time. In order to use the kernel module utilities described in this chapter, first ensure the kmod package is installed on your system by running, as root:

# yum install kmod

1.1. What is a kernel module?

The Linux kernel is monolithic by design. However, it is compiled with optional or additional modules as required by each use case. This means that you can extend the kernel’s capabilities through the use of dynamically-loaded kernel modules . A kernel module can provide: A device driver which adds support for new hardware. Support for a file system such as GFS2 or NFS . Like the kernel itself, modules can take parameters that customize their behavior. Though the default parameters work well in most cases. In relation to kernel modules, user-space tools can do the following operations: Listing modules currently loaded into a running kernel. Querying all available modules for available parameters and module-specific information. Loading or unloading (removing) modules dynamically into or from a running kernel. Many of these utilities, which are provided by the kmod package, take module dependencies into account when performing operations. As a result, manual dependency-tracking is rarely necessary. On modern systems, kernel modules are automatically loaded by various mechanisms when needed. However, there are occasions when it is necessary to load or unload modules manually. For example, when one module is preferred over another although either is able to provide basic functionality, or when a module performs unexpectedly.

1.2. Kernel module dependencies

Certain kernel modules sometimes depend on one or more other kernel modules. The /lib/modules/<KERNEL_VERSION>/modules.dep file contains a complete list of kernel module dependencies for the respective kernel version. The dependency file is generated by the depmod program, which is a part of the kmod package. Many of the utilities provided by kmod take module dependencies into account when performing operations so that manual dependency-tracking is rarely necessary.

Warning

The code of kernel modules is executed in kernel-space in the unrestricted mode. Because of this, you should be mindful of what modules you are loading.

Additional resources

For more information about /lib/modules/<KERNEL_VERSION>/modules.dep , refer to the modules.dep(5) manual page. For further details including the synopsis and options of depmod , see the depmod(8) manual page.

1.3. Listing currently-loaded modules

You can list all kernel modules that are currently loaded into the kernel by running the lsmod command, for example:

# lsmod
Module                  Size  Used by
tcp_lp                 12663  0
bnep                   19704  2
bluetooth             372662  7 bnep
rfkill                 26536  3 bluetooth
fuse                   87661  3
ebtable_broute         12731  0
bridge                110196  1 ebtable_broute
stp                    12976  1 bridge
llc                    14552  2 stp,bridge
ebtable_filter         12827  0
ebtables               30913  3 ebtable_broute,ebtable_nat,ebtable_filter
ip6table_nat           13015  1
nf_nat_ipv6            13279  1 ip6table_nat
iptable_nat            13011  1
nf_conntrack_ipv4      14862  4
nf_defrag_ipv4         12729  1 nf_conntrack_ipv4
nf_nat_ipv4            13263  1 iptable_nat
nf_nat                 21798  4 nf_nat_ipv4,nf_nat_ipv6,ip6table_nat,iptable_nat
[output truncated]

The lsmod output specifies three columns: Module The name of a kernel module currently loaded in memory. The amount of memory the kernel module uses in kilobytes. Used by A decimal number representing how many dependencies there are on the Module field. A comma separated string of dependent Module names. Using this list, you can first unload all the modules depending on the module you want to unload. Finally, note that lsmod output is less verbose and considerably easier to read than the content of the /proc/modules pseudo-file.

1.4. Displaying information about a module

You can display detailed information about a kernel module using the modinfo <MODULE_NAME> command. When entering the name of a kernel module as an argument to one of the kmod utilities, do not append a .ko extension to the end of the name. Kernel module names do not have extensions; their corresponding files do.

Example 1.1. Listing information about a kernel module with lsmod

To display information about the e1000e module, which is the Intel PRO/1000 network driver, enter the following command as root :

# modinfo e1000e
filename:       /lib/modules/3.10.0-121.el7.x86_64/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko
version:        2.3.2-k
license:        GPL
description:    Intel(R) PRO/1000 Network Driver
author:         Intel Corporation,

1.5. Loading kernel modules at system runtime

The optimal way to expand the functionality of the Linux kernel is by loading kernel modules. The following procedure describes how to use the modprobe command to find and load a kernel module into the currently running kernel.

Prerequisites

Root permissions. The kmod package is installed. The respective kernel module is not loaded. To ensure this is the case, see Listing Currently Loaded Modules .

Procedure

Select a kernel module you want to load. The modules are located in the /lib/modules/$(uname -r)/kernel/<SUBSYSTEM>/ directory. Load the relevant kernel module:
```
# modprobe <MODULE_NAME>
```
Note
When entering the name of a kernel module, do not append the .ko.xz extension to the end of the name. Kernel module names do not have extensions; their corresponding files do. Optionally, verify the relevant module was loaded:
```
$ lsmod | grep <MODULE_NAME>
```
If the module was loaded correctly, this command displays the relevant kernel module. For example:
```
$ lsmod | grep serio_raw
serio_raw              16384  0
```

Important

The changes described in this procedure will not persist after rebooting the system. For information on how to load kernel modules to persist across system reboots, see Loading kernel modules automatically at system boot time .

Additional resources

For further details about modprobe , see the modprobe(8) manual page.

1.6. Unloading kernel modules at system runtime

At times, you find that you need to unload certain kernel modules from the running kernel. The following procedure describes how to use the modprobe command to find and unload a kernel module at system runtime from the currently loaded kernel.

Prerequisites

Root permissions. The kmod package is installed.

Procedure

Execute the lsmod command and select a kernel module you want to unload. If a kernel module has dependencies, unload those prior to unloading the kernel module. For details on identifying modules with dependencies, see Listing Currently Loaded Modules and Kernel module dependencies . Unload the relevant kernel module:
```
# modprobe -r <MODULE_NAME>
```
When entering the name of a kernel module, do not append the .ko.xz extension to the end of the name. Kernel module names do not have extensions; their corresponding files do.
Warning
Do not unload kernel modules when they are used by the running system. Doing so can lead to an unstable or non-operational system. Optionally, verify the relevant module was unloaded:
```
$ lsmod | grep <MODULE_NAME>
```
If the module was unloaded successfully, this command does not display any output.

Important

After finishing this procedure, the kernel modules that are defined to be automatically loaded on boot, will not stay unloaded after rebooting the system. For information on how to counter this outcome, see Preventing kernel modules from being automatically loaded at system boot time .

Additional resources

For further details about modprobe , see the modprobe(8) manual page.

1.7. Loading kernel modules automatically at system boot time

The following procedure describes how to configure a kernel module so that it is loaded automatically during the boot process.

Prerequisites

Root permissions. The kmod package is installed.

Procedure

Select a kernel module you want to load during the boot process. The modules are located in the /lib/modules/$(uname -r)/kernel/<SUBSYSTEM>/ directory. Create a configuration file for the module:
```
# echo <MODULE_NAME> > /etc/modules-load.d/<MODULE_NAME>.conf
```
Note
When entering the name of a kernel module, do not append the .ko.xz extension to the end of the name. Kernel module names do not have extensions; their corresponding files do. Optionally, after reboot, verify the relevant module was loaded:
```
$ lsmod | grep <MODULE_NAME>
```
The example command above should succeed and display the relevant kernel module.

Important

The changes described in this procedure will persist after rebooting the system.

Additional resources

For further details about loading kernel modules during the boot process, see the modules-load.d(5) manual page.

1.8. Preventing kernel modules from being automatically loaded at system boot time

The following procedure describes how to add a kernel module to a denylist so that it will not be automatically loaded during the boot process.

Prerequisites

Root permissions. The kmod package is installed. Ensure that a kernel module in a denylist is not vital for your current system configuration.

Procedure

Select a kernel module that you want to put in a denylist:

$ lsmod Module Size Used by fuse 126976 3 xt_CHECKSUM 16384 1 ipt_MASQUERADE 16384 1 uinput 20480 1 xt_conntrack 16384 1 The lsmod command displays a list of modules loaded to the currently running kernel. Alternatively, identify an unloaded kernel module you want to prevent from potentially loading. All kernel modules are located in the /lib/modules/<KERNEL_VERSION>/kernel/<SUBSYSTEM>/ directory. Create a configuration file for a denylist:

# vim /etc/modprobe.d/blacklist.conf # Blacklists <KERNEL_MODULE_1> blacklist <MODULE_NAME_1> install <MODULE_NAME_1> /bin/false # Blacklists <KERNEL_MODULE_2> blacklist <MODULE_NAME_2> install <MODULE_NAME_2> /bin/false # Blacklists <KERNEL_MODULE_n> blacklist <MODULE_NAME_n> install <MODULE_NAME_n> /bin/false The example shows the contents of the blacklist.conf file, edited by the vim editor. The blacklist line ensures that the relevant kernel module will not be automatically loaded during the boot process. The blacklist command, however, does not prevent the module from being loaded as a dependency for another kernel module that is not in a denylist. Therefore the install line causes the /bin/false to run instead of installing a module. The lines starting with a hash sign are comments to make the file more readable. When entering the name of a kernel module, do not append the .ko.xz extension to the end of the name. Kernel module names do not have extensions; their corresponding files do. Create a backup copy of the current initial ramdisk image before rebuilding:

# cp /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r).bak.$(date +%m-%d-%H%M%S).img

The command above creates a backup initramfs image in case the new version has an unexpected problem. Alternatively, create a backup copy of other initial ramdisk image which corresponds to the kernel version for which you want to put kernel modules in a denylist:

# cp /boot/initramfs-<SOME_VERSION>.img /boot/initramfs-<SOME_VERSION>.img.bak.$(date +%m-%d-%H%M%S)

Generate a new initial ramdisk image to reflect the changes:

# dracut -f -v

If you are building an initial ramdisk image for a different kernel version than you are currently booted into, specify both target initramfs and kernel version:
```
# dracut -f -v /boot/initramfs-<TARGET_VERSION>.img <CORRESPONDING_TARGET_KERNEL_VERSION>
```

Reboot the system:

$ reboot

Important

The changes described in this procedure will take effect and persist after rebooting the system. If you improperly put a key kernel module in a denylist, you can face an unstable or non-operational system.

Additional resources

For further details concerning the dracut utility, refer to the dracut(8) manual page. For more information on preventing automatic loading of kernel modules at system boot time on Red Hat Enterprise Linux 8 and earlier versions, see How do I prevent a kernel module from loading automatically?

1.9. Signing kernel modules for secure boot

Red Hat Enterprise Linux 7 includes support for the UEFI Secure Boot feature, which means that Red Hat Enterprise Linux 7 can be installed and run on systems where UEFI Secure Boot is enabled. Note that Red Hat Enterprise Linux 7 does not require the use of Secure Boot on UEFI systems. If Secure Boot is enabled, the UEFI operating system boot loaders, the Red Hat Enterprise Linux kernel, and all kernel modules must be signed with a private key and authenticated with the corresponding public key. If they are not signed and authenticated, the system will not be allowed to finish the booting process. The Red Hat Enterprise Linux 7 distribution includes: Signed boot loaders Signed kernels Signed kernel modules In addition, the signed first-stage boot loader and the signed kernel include embedded Red Hat public keys. These signed executable binaries and embedded keys enable Red Hat Enterprise Linux 7 to install, boot, and run with the Microsoft UEFI Secure Boot Certification Authority keys that are provided by the UEFI firmware on systems that support UEFI Secure Boot. Not all UEFI-based systems include support for Secure Boot. The information provided in the following sections describes the steps to self-sign privately built kernel modules for use with Red Hat Enterprise Linux 7 on UEFI-based build systems where Secure Boot is enabled. These sections also provide an overview of available options for importing your public key into a target system where you want to deploy your kernel modules. To sign and load kernel modules, you need to: Have the relevant utilities installed on your system . Authenticate a kernel module . Generate a public and private key pair . Import the public key on the target system . Sign the kernel module with the private key . Load the signed kernel module .

1.9.1. Prerequisites

To be able to sign externally built kernel modules, install the utilities listed in the following table on the build system.

Table 1.1. Required utilities

Utility Provided by package Used on Purpose

openssl openssl Build system Generates public and private X.509 key pair sign-file kernel-devel Build system Perl script used to sign kernel modules Build system Perl interpreter used to run the signing script mokutil mokutil Target system Optional utility used to manually enroll the public key keyctl keyutils Target system Optional utility used to display public keys in the system key ring The build system, where you build and sign your kernel module, does not need to have UEFI Secure Boot enabled and does not even need to be a UEFI-based system.

1.9.2. Kernel module authentication

In Red Hat Enterprise Linux 7, when a kernel module is loaded, the module’s signature is checked using the public X.509 keys on the kernel’s system key ring, excluding keys on the kernel’s system black-list key ring. The following sections provide an overview of sources of keys/keyrings, examples of loaded keys from different sources in the system. Also, the user can see what it takes to authenticate a kernel module.

1.9.2.1. Sources for public keys used to authenticate kernel modules

During boot, the kernel loads X.509 keys into the system key ring or the system black-list key ring from a set of persistent key stores as shown in the table below.

Table 1.2. Sources for system key rings

Source of X.509 keys User ability to add keys UEFI Secure Boot state Keys loaded during boot

Embedded in kernel .system_keyring UEFI Secure Boot "db" Limited Not enabled Enabled .system_keyring UEFI Secure Boot "dbx" Limited Not enabled Enabled .system_keyring Embedded in shim.efi boot loader Not enabled Enabled .system_keyring Machine Owner Key (MOK) list Not enabled Enabled .system_keyring If the system is not UEFI-based or if UEFI Secure Boot is not enabled, then only the keys that are embedded in the kernel are loaded onto the system key ring. In that case you have no ability to augment that set of keys without rebuilding the kernel. The system black list key ring is a list of X.509 keys which have been revoked. If your module is signed by a key on the black list then it will fail authentication even if your public key is in the system key ring. You can display information about the keys on the system key rings using the keyctl utility. The following is a shortened example output from a Red Hat Enterprise Linux 7 system where UEFI Secure Boot is not enabled.

# keyctl list %:.system_keyring
3 keys in keyring:
...asymmetric: Red Hat Enterprise Linux Driver Update Program (key 3): bf57f3e87...
...asymmetric: Red Hat Enterprise Linux kernel signing key: 4249689eefc77e95880b...
...asymmetric: Red Hat Enterprise Linux kpatch signing key: 4d38fd864ebe18c5f0b7...

The following is a shortened example output from a Red Hat Enterprise Linux 7 system where UEFI Secure Boot is enabled.

# keyctl list %:.system_keyring
6 keys in keyring:
...asymmetric: Red Hat Enterprise Linux Driver Update Program (key 3): bf57f3e87...
...asymmetric: Red Hat Secure Boot (CA key 1): 4016841644ce3a810408050766e8f8a29...
...asymmetric: Microsoft Corporation UEFI CA 2011: 13adbf4309bd82709c8cd54f316ed...
...asymmetric: Microsoft Windows Production PCA 2011: a92902398e16c49778cd90f99e...
...asymmetric: Red Hat Enterprise Linux kernel signing key: 4249689eefc77e95880b...
...asymmetric: Red Hat Enterprise Linux kpatch signing key: 4d38fd864ebe18c5f0b7...

The above output shows the addition of two keys from the UEFI Secure Boot "db" keys as well as the Red Hat Secure Boot (CA key 1) , which is embedded in the shim.efi boot loader. You can also look for the kernel console messages that identify the keys with an UEFI Secure Boot related source. These include UEFI Secure Boot db, embedded shim, and MOK list.

# dmesg | grep 'EFI: Loaded cert'
[5.160660] EFI: Loaded cert 'Microsoft Windows Production PCA 2011: a9290239...
[5.160674] EFI: Loaded cert 'Microsoft Corporation UEFI CA 2011: 13adbf4309b...
[5.165794] EFI: Loaded cert 'Red Hat Secure Boot (CA key 1): 4016841644ce3a8...

1.9.2.2. Kernel module authentication requirements

This section explains what conditions have to be met for loading kernel modules on systems with enabled UEFI Secure Boot functionality. If UEFI Secure Boot is enabled or if the module.sig_enforce kernel parameter has been specified, you can only load signed kernel modules that are authenticated using a key on the system key ring. In addition, the public key must not be on the system black list key ring. If UEFI Secure Boot is disabled and if the module.sig_enforce kernel parameter has not been specified, you can load unsigned kernel modules and signed kernel modules without a public key. This is summarized in the table below.

Table 1.3. Kernel module authentication requirements for loading

Module signed	Public key found and signature valid	UEFI Secure Boot state	sig_enforce	Module load	Kernel tainted
Unsigned Not enabled Not enabled Succeeds Not enabled Enabled Fails Enabled Fails Signed Not enabled Not enabled Succeeds Not enabled Enabled Fails Enabled Fails Signed Not enabled Not enabled Succeeds Not enabled Enabled Succeeds Enabled Succeeds

1.9.3. Generating a public and private X.509 key pair

You need to generate a public and private X.509 key pair to succeed in your efforts of using kernel modules on a Secure Boot-enabled system. You will later use the private key to sign the kernel module. You will also have to add the corresponding public key to the Machine Owner Key (MOK) for Secure Boot to validate the signed module. For instructions to do so, see Section 1.9.4.2, “System administrator manually adding public key to the MOK list” . Some of the parameters for this key pair generation are best specified with a configuration file. Create a configuration file with parameters for the key pair generation:

# cat << EOF > configuration_file.config
[ req ]
default_bits = 4096
distinguished_name = req_distinguished_name
prompt = no
string_mask = utf8only
x509_extensions = myexts
[ req_distinguished_name ]
O = Organization
CN = Organization signing key
emailAddress = E-mail address
[ myexts ]
basicConstraints=critical,CA:FALSE
keyUsage=digitalSignature
subjectKeyIdentifier=hash
authorityKeyIdentifier=keyid
							Create an X.509 public and private key pair as shown in the following example:
						
# openssl req -x509 -new -nodes -utf8 -sha256 -days 36500 \
-batch -config configuration_file.config -outform DER \
-out my_signing_key_pub.der \
-keyout my_signing_key.priv

							The public key will be written to the my_signing_key_pub.der file and the private key will be written to the my_signing_key.priv file.
							Enroll your public key on all systems where you want to authenticate and load your kernel module.
							For details, see Section 1.9.4, “Enrolling public key on target system”.
						
Warning

						Apply strong security measures and access policies to guard the contents of your private key. In the wrong hands, the key could be used to compromise any system which is authenticated by the corresponding public key.

1.9.4. Enrolling public key on target system

When Red Hat Enterprise Linux 7 boots on a UEFI-based system with Secure Boot enabled, the kernel loads onto the system key ring all public keys that are in the Secure Boot db key database, but not in the dbx database of revoked keys. The sections below describe different ways of importing a public key on a target system so that the system key ring is able to use the public key to authenticate a kernel module.

1.9.4.1. Factory firmware image including public key

To facilitate authentication of your kernel module on your systems, consider requesting your system vendor to incorporate your public key into the UEFI Secure Boot key database in their factory firmware image.

1.9.4.2. System administrator manually adding public key to the MOK list

The Machine Owner Key (MOK) facility feature can be used to expand the UEFI Secure Boot key database. When Red Hat Enterprise Linux 7 boots on a UEFI-enabled system with Secure Boot enabled, the keys on the MOK list are also added to the system key ring in addition to the keys from the key database. The MOK list keys are also stored persistently and securely in the same fashion as the Secure Boot database keys, but these are two separate facilities. The MOK facility is supported by shim.efi , MokManager.efi , grubx64.efi , and the Red Hat Enterprise Linux 7 mokutil utility. Enrolling a MOK key requires manual interaction by a user at the UEFI system console on each target system. Nevertheless, the MOK facility provides a convenient method for testing newly generated key pairs and testing kernel modules signed with them. To add your public key to the MOK list: Request the addition of your public key to the MOK list:

# mokutil --import my_signing_key_pub.der

You will be asked to enter and confirm a password for this MOK enrollment request. Reboot the machine. The pending MOK key enrollment request will be noticed by shim.efi and it will launch MokManager.efi to allow you to complete the enrollment from the UEFI console. Enter the password you previously associated with this request and confirm the enrollment. Your public key is added to the MOK list, which is persistent. Once a key is on the MOK list, it will be automatically propagated to the system key ring on this and subsequent boots when UEFI Secure Boot is enabled.

1.9.5. Signing kernel module with the private key

Assuming you have your kernel module ready: Use a Perl script to sign your kernel module with your private key:

# perl /usr/src/kernels/$(uname -r)/scripts/sign-file \
sha256 \
my_signing_key.priv\
my_signing_key_pub.der\
my_module.ko

Note

The Perl script requires that you provide both the files that contain your private and the public key as well as the kernel module file that you want to sign. Your kernel module is in ELF image format and the Perl script computes and appends the signature directly to the ELF image in your kernel module file. The modinfo utility can be used to display information about the kernel module’s signature, if it is present. For information on using modinfo , see Section 1.4, “Displaying information about a module” . The appended signature is not contained in an ELF image section and is not a formal part of the ELF image. Therefore, utilities such as readelf will not be able to display the signature on your kernel module. Your kernel module is now ready for loading. Note that your signed kernel module is also loadable on systems where UEFI Secure Boot is disabled or on a non-UEFI system. That means you do not need to provide both a signed and unsigned version of your kernel module.

1.9.6. Loading signed kernel module

Once your public key is enrolled and is in the system key ring, use mokutil to add your public key to the MOK list. Then manually load your kernel module with the modprobe command. Optionally, verify that your kernel module will not load before you have enrolled your public key. For details on how to list currently loaded kernel modules, see Section 1.3, “Listing currently-loaded modules” . Verify what keys have been added to the system key ring on the current boot:

# keyctl list %:.system_keyring

Since your public key has not been enrolled yet, it should not be displayed in the output of the command. Request enrollment of your public key:

# mokutil --import my_signing_key_pub.der

Reboot, and complete the enrollment at the UEFI console:

# reboot

Verify the keys on the system key ring again:

# keyctl list %:.system_keyring

Copy the module into the /extra/ directory of the kernel you want:

# cp my_module.ko /lib/modules/$(uname -r)/extra/

Update the modular dependency list:

# depmod -a

Load the kernel module and verify that it was successfully loaded:

# modprobe -v my_module
# lsmod | grep my_module

Optionally, to load the module on boot, add it to the /etc/modules-loaded.d/my_module.conf file:
```
# echo "my_module" > /etc/modules-load.d/my_module.conf
```

Chapter 2. Working with sysctl and kernel tunables

2.1. What is a kernel tunable?

Kernel tunables are used to customize the behavior of Red Hat Enterprise Linux at boot, or on demand while the system is running. Some hardware parameters are specified at boot time only and cannot be altered once the system is running, most however, can be altered as required and set permanent for the next boot.

2.2. How to work with kernel tunables

There are three ways to modify kernel tunables. Using the sysctl command By manually modifying configuration files in the /etc/sysctl.d/ directory Through a shell, interacting with the virtual file system mounted at /proc/sys Not all boot time parameters are under control of the sysfs subsystem, some hardware specific option must be set on the kernel command line, the Kernel Parameters section of this guide addresses those options

2.2.1. Using the sysctl command

The sysctl command is used to list, read, and set kernel tunables. It can filter tunables when listing or reading and set tunables temporarily or permanently. Listing variables

# sysctl -a

Reading variables

# sysctl kernel.version
kernel.version = #1 SMP Fri Jan 19 13:19:54 UTC 2018

Writing variables temporarily

# sysctl <tunable class>.<tunable>=<value>

Writing variables permanently

# sysctl -w <tunable class>.<tunable>=<value> >> /etc/sysctl.conf

2.2.2. Modifying files in /etc/sysctl.d

To override a default at boot, you can also manually populate files in /etc/sysctl.d . Create a new file in /etc/sysctl.d

# vim /etc/sysctl.d/99-custom.conf

Include the variables you wish to set, one per line, in the following form

<tunable class>.<tunable> = <value> +
<tunable class>.<tunable> = <value>

Save the file Either reboot the machine to make the changes take effect
or
Execute


                       sysctl -p /etc/sysctl.d/99-custom.conf

to apply the changes without rebooting

2.3. What tunables can be controlled?

Tunables are divided into groups by kernel sybsystem. A Red Hat Enterprise Linux system has the following classes of tunables:

Table 2.1. Table of sysctl interfaces

Class	Subsystem
Execution domains and personalities crypto Cryptographic interfaces debug Kernel debugging interfaces Device specific information Global and specific filesystem tunables kernel Global kernel tunables Network tunables sunrpc Sun Remote Procedure Call (NFS) User Namespace limits Tuning and management of memory, buffer, and cache

2.3.1. Network interface tunables

System administrators are able to adjust the network configuration on a running system through the networking tunables. Networking tunables are included in the /proc/sys/net directory, which contains multiple subdirectories for various networking topics. To adjust the network configuration, system administrators need to modify the files within such subdirectories. The most frequently used directories are: /proc/sys/net/core/ /proc/sys/net/ipv4/ The /proc/sys/net/core/ directory contains a variety of settings that control the interaction between the kernel and networking layers. By adjusting some of those tunables, you can improve performance of a system, for example by increasing the size of a receive queue, increasing the maximum connections or the memory dedicated to network interfaces. Note that the performance of a system depends on different aspects according to the individual issues. The /proc/sys/net/ipv4/ directory contains additional networking settings, which are useful when preventing attacks on the system or when using the system to act as a router. The directory contains both IP and TCP variables. For detailed explaination of those variables, see /usr/share/doc/kernel-doc-<version>/Documentation/networking/ip-sysctl.txt . Other directories within the /proc/sys/net/ipv4/ directory cover different aspects of the network stack: /proc/sys/net/ipv4/conf/ - allows you to configure each system interface in different ways, including the use of default settings for unconfigured devices and settings that override all special configurations /proc/sys/net/ipv4/neigh/ - contains settings for communicating with a host directly connected to the system and also contains different settings for systems more than one step away /proc/sys/net/ipv4/route/ - contains specifications that apply to routing with any interfaces on the system This list of network tunables is relevant to IPv4 interfaces and are accessible from the /proc/sys/net/ipv4/{all,<interface_name>}/ directory. Description of the following parameters have been adopted from the kernel documentation sites. ^[1]

log_martians

Log packets with impossible addresses to kernel log.

Type Default

Boolean Enabled if one or more of conf/{all,interface}/log_martians is set to TRUE Further Resources What is the kernel parameter net.ipv4.conf.all.log_martians for? Why do I see "martian source" logs in the messages file ?

accept_redirects

Accept ICMP redirect messages.

Type Default

Boolean accept_redirects for the interface is enabled under the following conditions: Both conf/{all,interface}/accept_redirects are TRUE (when forwarding for the interface is enabled) At least one of conf/{all,interface}/accept_redirects is TRUE (forwarding for the interface is disabled) For more information refer to How to enable or disable ICMP redirects

forwarding

Enable IP forwarding on an interface.

Type Default

Boolean Further Resources Turning on Packet Forwarding and Nonlocal Binding

mc_forwarding

Do multicast routing.

Type Default

Boolean Read only value A multicast routing daemon is required. conf/all/mc_forwarding must also be set to TRUE to enable multicast routing for the interface Further Resources For an explanation of the read only behavior, see Why system reports "permission denied on key" while setting the kernel parameter "net.ipv4.conf.all.mc_forwarding"?

medium_id

Arbitrary value used to differentiate the devices by the medium they are attached to.

Type Default

Integer Notes Two devices on the same medium can have different id values when the broadcast packets are received only on one of them. The default value 0 means that the device is the only interface to its medium value of -1 means that medium is not known. Currently, it is used to change the proxy_arp behavior: the proxy_arp feature is enabled for packets forwarded between two devices attached to different media. Further Resources - For examples, see Using the "medium_id" feature in Linux 2.2 and 2.4

proxy_arp

Do proxy arp.

Type Default

Boolean proxy_arp for the interface is enabled if at least one of conf/{all,interface}/proxy_arp is set to TRUE, otherwise it is disabled

proxy_arp_pvlan

Private VLAN proxy arp.

Type Default

Boolean Allow proxy arp replies back to the same interface, to support features like RFC 3069

shared_media

Send(router) or accept(host) RFC1620 shared media redirects.

Type Default

Boolean Notes
Overrides secure_redirects. shared_media for the interface is enabled if at least one of conf/{all,interface}/shared_media is set to TRUE

secure_redirects

Accept ICMP redirect messages only to gateways listed in the interface’s current gateway list.

Type Default

Boolean Notes Even if disabled, RFC1122 redirect rules still apply. Overridden by shared_media. secure_redirects for the interface is enabled if at least one of conf/{all,interface}/secure_redirects is set to TRUE

send_redirects

Send redirects, if router.

Type Default

Boolean Notes
send_redirects for the interface is enabled if at least one of conf/{all,interface}/send_redirects is set to TRUE

bootp_relay

Accept packets with source address 0.b.c.d destined not to this host as local ones.

Type Default

Boolean Notes
A BOOTP daemon must be enabled to manage these packets conf/all/bootp_relay must also be set to TRUE to enable BOOTP relay for the interface Not implemented, see DHCP Relay Agent in the Red Hat Enterprise Linux Networking Guide

accept_source_route

Accept packets with SRR option.

Type Default

Boolean Notes
conf/all/accept_source_route must also be set to TRUE to accept packets with SRR option on the interface

accept_local

Accept packets with local source addresses.

Type Default

Boolean Notes
In combination with suitable routing, this can be used to direct packets between two local interfaces over the wire and have them accepted properly. rp_filter must be set to a non-zero value in order for accept_local to have an effect.

route_localnet

Do not consider loopback addresses as martian source or destination while routing.

Type Default

Boolean Notes
This enables the use of 127/8 for local routing purposes.

rp_filter

Enable source Validation

Type	Default
Integer

Value Effect

No source validation. Strict mode as defined in RFC3704 Strict Reverse Path Loose mode as defined in RFC3704 Loose Reverse Path Notes
Current recommended practice in RFC3704 is to enable strict mode to prevent IP spoofing from DDos attacks. If using asymmetric routing or other complicated routing, then loose mode is recommended. The highest value from conf/{all,interface}/rp_filter is used when doing source validation on the {interface}

arp_filter

Type	Default
Boolean

Value Effect

(default) The kernel can respond to arp requests with addresses from other interfaces. It usually makes sense, because it increases the chance of successful communication. Allows you to have multiple network interfaces on the samesubnet, and have the ARPs for each interface be answered based on whether or not the kernel would route a packet from the ARP’d IP out that interface (therefore you must use source based routing for this to work). In other words it allows control of cards (usually 1) that respond to an arp request. IP addresses are owned by the complete host on Linux, not by particular interfaces. Only for more complex setups like load-balancing, does this behavior cause problems. arp_filter for the interface is enabled if at least one of conf/{all,interface}/arp_filter is set to TRUE

arp_announce

Define different restriction levels for announcing the local source IP address from IP packets in ARP requests sent on interface

Type	Default
Integer

Value Effect

(default) Use any local address, configured on any interface Try to avoid local addresses that are not in the target’s subnet for this interface. This mode is useful when target hosts reachable via this interface require the source IP address in ARP requests to be part of their logical network configured on the receiving interface. When we generate the request we check all our subnets that include the target IP and preserve the source address if it is from such subnet. If there is no such subnet we select source address according to the rules for level 2. Always use the best local address for this target. In this mode we ignore the source address in the IP packet and try to select local address that we prefer for talks with the target host. Such local address is selected by looking for primary IP addresses on all our subnets on the outgoing interface that include the target IP address. If no suitable local address is found we select the first local address we have on the outgoing interface or on all other interfaces, with the hope we receive reply for our request and even sometimes no matter the source IP address we announce. Notes The highest value from conf/{all,interface}/arp_announce is used. Increasing the restriction level gives more chance for receiving answer from the resolved target while decreasing the level announces more valid sender’s information.

arp_ignore

Define different modes for sending replies in response to received ARP requests that resolve local target IP addresses

Type	Default
Integer

Value Effect

(default): reply for any local target IP address, configured on any interface reply only if the target IP address is local address configured on the incoming interface reply only if the target IP address is local address configured on the incoming interface and both with the sender’s IP address are part from same subnet on this interface do not reply for local addresses configured with scope host, only resolutions for global and link addresses are replied reserved do not reply for all local addresses The max value from conf/{all,interface}/arp_ignore is used when ARP request is received on the {interface} Notes

arp_notify

Define mode for notification of address and device changes.

Type	Default
Boolean

Value Effect

do nothing Generate gratuitous arp requests when device is brought up or hardware address changes. Notes

arp_accept

Define behavior for gratuitous ARP frames who’s IP is not already present in the ARP table

Type	Default
Boolean

Value Effect

do not create new entries in the ARP table create new entries in the ARP table. Notes
Both replies and requests type gratuitous arp trigger the ARP table to be updated, if this setting is on. If the ARP table already contains the IP address of the gratuitous arp frame, the arp table is updated regardless if this setting is on or off.

app_solicit

The maximum number of probes to send to the user space ARP daemon via netlink before dropping back to multicast probes (see mcast_solicit).

Type Default

Integer Notes
See mcast_solicit

disable_policy

Disable IPSEC policy (SPD) for this interface

Type Default

Boolean needinfo

disable_xfrm

Disable IPSEC encryption on this interface, whatever the policy

Type Default

Boolean needinfo

igmpv2_unsolicited_report_interval

The interval in milliseconds in which the next unsolicited IGMPv1 or IGMPv2 report retransmit takes place.

Type Default

Integer 10000 Notes
Milliseconds

igmpv3_unsolicited_report_interval

The interval in milliseconds in which the next unsolicited IGMPv3 report retransmit takes place.

Type Default

Integer Notes
Milliseconds Allows you to write a number, which can be used as required.

Type	Default
Integer

xfrm4_gc_thresh

The threshold at which we start garbage collecting for IPv4 destination cache entries.

Type Default

Integer Notes
At twice this value the system refuses new allocations.

2.3.2. Global kernel tunables

System administrators are able to configure and monitor general settings on a running system through the global kernel tunables. Global kernel tunables are included in the /proc/sys/kernel/ directory either directly as named control files or grouped in further subdirectories for various configuration topics. To adjust the global kernel tunables, system administrators need to modify the control files. Descriptions of the following parameters have been adopted from the kernel documentation sites. ^[2]

dmesg_restrict: Indicates whether unprivileged users are prevented from using the dmesg command to view messages from the kernel’s log buffer. For further information, see Kernel sysctl documentation .

core_pattern

Specifies a core dumpfile pattern name.

Max length Default

128 characters "core" For further information, see Kernel sysctl documentation .

hardlockup_panic

Controls the kernel panic when a hard lockup is detected.

Type Value Effect

Integer kernel does not panic on hard lockup Integer kernel panics on hard lockup In order to panic, the system needs to detect a hard lockup first. The detection is controlled by the nmi_watchdog parameter. Further Resources Kernel sysctl documentation Softlockup detector and hardlockup detector

softlockup_panic

Controls the kernel panic when a soft lockup is detected.

Type	Value	Effect
Integer kernel does not panic on soft lockup Integer kernel panics on soft lockup By default, on RHEL7 this value is 0. For more information about `softlockup_panic` , see kernel_parameters .

Using modules, kpatch, or kdump to manage Linux kernel in RHEL

Jaroslav Klech

Sujata Kurup

Khushbu Borole

Marie Dolezelova

Mark Flitter

Douglas Silas

Eliska Slobodova

Jaromir Hradilek

Maxim Svistunov

Robert Krátký

Stephen Wadeley

Florian Nadge

Preface

Chapter 1. Working with kernel modules

1.1. What is a kernel module?

1.2. Kernel module dependencies

1.3. Listing currently-loaded modules

1.4. Displaying information about a module

1.5. Loading kernel modules at system runtime

1.6. Unloading kernel modules at system runtime

1.7. Loading kernel modules automatically at system boot time

1.8. Preventing kernel modules from being automatically loaded at system boot time

1.9. Signing kernel modules for secure boot

1.9.1. Prerequisites

1.9.2. Kernel module authentication

1.9.2.1. Sources for public keys used to authenticate kernel modules

1.9.2.2. Kernel module authentication requirements

1.9.3. Generating a public and private X.509 key pair

1.9.4. Enrolling public key on target system

1.9.4.1. Factory firmware image including public key

1.9.4.2. System administrator manually adding public key to the MOK list

1.9.5. Signing kernel module with the private key

1.9.6. Loading signed kernel module

Chapter 2. Working with sysctl and kernel tunables

2.1. What is a kernel tunable?

2.2. How to work with kernel tunables

2.2.1. Using the sysctl command

2.2.2. Modifying files in /etc/sysctl.d

2.3. What tunables can be controlled?

2.3.1. Network interface tunables

2.3.2. Global kernel tunables