MDEV-21935 Mariabackup file limits set incorrectly #4612
Conversation
|
assigned for review for server team / @sanja-byelkin . I could not figure out a safe deterministic mtr test for this. Tested only manually by starting a 2 node cluster (by mariabackup SST) and verifying that mariabackup execution had ulimit set to according to open-files-limit as configured for mariabackup |
janlindstrom
left a comment
janlindstrom
left a comment
There was a problem hiding this comment.
Please check indentation and soft limit case.
| } | ||
|
|
||
| /* Soft limit cannot exceed hard limit. */ | ||
| rlimit.rlim_cur= max_file_limit; |
There was a problem hiding this comment.
If setting new limit fails, does it also mean that soft limit remains bigger than hard limit ?
There was a problem hiding this comment.
hard limit is the ceiling, soft limit can never been higher
mysys/my_file.c
Outdated
| /* Never lower the hard limit (rlim_max): lowering is irreversible for | ||
| non-privileged users and can prevent later increases. */ | ||
| if ((ulonglong) rlimit.rlim_max != (ulonglong) RLIM_INFINITY && | ||
| max_file_limit > rlimit.rlim_max) |
There was a problem hiding this comment.
Indentation is incorrect
There was a problem hiding this comment.
original file has tabs here and there, replaced by spaces
When a cluster donor node executes mariabackup SST, it will use same approximation for max open files limit as was set for the mariadbd process during the server startup. This may be a problem for installation where mariabackup execution would need higher open file count, and might crash for exceeding the too tight open file limit. The reason for this behavior is that when mariadbd server calculates the expected max open files count, it will record this open file count approximation as system ulimit value, both as soft and hard limit. Later, when the node operates as SST donor, it spawns mariabackup executable, which now inherits the ulmit setting used for the mariadbd process. Mariabackup tries to enforce open_files_limit variable value configured in [mariabackup] group in the my.cnf fle, but this will fail if hard ulimit value is smaller. The fix in this commit records the approximated max open file count only as soft ulimit value. If hard ulimit is higher or unlimited, there remains head room for the mariabackup to use higher open_files_limit configuration.
sjaakola
force-pushed
the
10.11-MDEV-21935
branch
from
f94cb94 to
1ef2782
Compare
|
Pushed new version based on Jan's review comments |
3 participants
When a cluster donor node executes mariabackup SST, it will use same approximation for max open files limit as was set for the mariadbd process during the server startup. This may be a problem for installation where mariabackup execution would need higher open file count, and might crash for exceeding the too tight open file limit.
The reason for this behavior is that when mariadbd server calculates the expected max open files count, it will record this open file count approximation as system ulimit value, both as soft and hard limit. Later, when the node operates as SST donor, it spawns mariabackup executable, which now inherits the ulmit setting used for the mariadbd process. Mariabackup tries to enforce open_files_limit variable value configured in [mariabackup] group in the my.cnf fle, but this will fail if hard ulimit value is smaller.
The fix in this commit records the approximated max open file count only as soft ulimit value. If hard ulimit is higher or unlimited, there remains head room for the mariabackup to use higher open_files_limit configuration.