Resource (CPU time and memory) limitation and termination of a process upon violation in Linux
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}
The problem:
Given a process, limit the resources it and its child-processes can use. I.e. set CPU time and virtual memory quotas. When the process group exceeds one of the limits, terminate it, otherwise print the amount of CPU time and virtual memory it has used.
The concrete use case:
Basically I must execute a couple of binaries, which expect input from a file, but I must ensure that their execution process is strictly limited. For example the binary must not allocate more than 256 MB of memory and it should run for less than 0.5 seconds. Hope this makes sense now. However I need information about the amount of memory and CPU it has used.
What I have tried:
- For a couple of days I have been dealing with this perl script, which is the best solution I have found so far. Unfortunately its memory is buggy and it is not very precise. Also there is an official author post about this script here.
- I have tried using both
/usr/bin/timeoutandtimeoutLinux tools, which of course help me with the CPU time quota, but not with the termination of the process due to violation of the virtual memory limit. - Using
ulimitwas attempted as well but as I have said earlier I need not only limitation but feedback for the resource consumption too.
The Question:
What can solve this issue? .
Edit 1: Added information for the concrete use case for better understanding of the problem.
process memory monitoring cpu cpu-usage
edited 3 hours ago
Rui F Ribeiro
41.9k1483142
asked Jul 15 '15 at 10:44
0x4500x450
1208
|
show 5 more comments
The problem:
Given a process, limit the resources it and its child-processes can use. I.e. set CPU time and virtual memory quotas. When the process group exceeds one of the limits, terminate it, otherwise print the amount of CPU time and virtual memory it has used.
The concrete use case:
Basically I must execute a couple of binaries, which expect input from a file, but I must ensure that their execution process is strictly limited. For example the binary must not allocate more than 256 MB of memory and it should run for less than 0.5 seconds. Hope this makes sense now. However I need information about the amount of memory and CPU it has used.
What I have tried:
- For a couple of days I have been dealing with this perl script, which is the best solution I have found so far. Unfortunately its memory is buggy and it is not very precise. Also there is an official author post about this script here.
- I have tried using both
/usr/bin/timeoutandtimeoutLinux tools, which of course help me with the CPU time quota, but not with the termination of the process due to violation of the virtual memory limit. - Using
ulimitwas attempted as well but as I have said earlier I need not only limitation but feedback for the resource consumption too.
The Question:
What can solve this issue? .
Edit 1: Added information for the concrete use case for better understanding of the problem.
process memory monitoring cpu cpu-usage
edited 3 hours ago
Rui F Ribeiro
41.9k1483142
asked Jul 15 '15 at 10:44
0x4500x450
1208
Your question is too broad IMHO. Could you motivate it more and give more details. So please edit your question to improve it. What is the software you have trouble with? Do you have its source code?
– Basile Starynkevitch
Jul 15 '15 at 11:15
Basically I must execute a couple of binaries, which expect input from a file, but I must ensure that their execution process is strictly limited. For example the binary must not allocate more than 256 MB of memory and it should run for less than 0.5 seconds. Hope this makes sense now. However I need information about the amount of memory and CPU it used.
– 0x450
Jul 15 '15 at 11:24
Please edit your question instead of commenting it. BTW, my answer fits (except thatsetrlimitcan limit CPU time only for an integral number of seconds)
– Basile Starynkevitch
Jul 15 '15 at 11:25
Can you raise the 0.5 CPU second limit to 1 second? If yes, usesetrlimit
– Basile Starynkevitch
Jul 15 '15 at 11:45
Are you able to code in C or any programming language giving access to Linux low level syscalls?
– Basile Starynkevitch
Jul 15 '15 at 11:49
|
show 5 more comments
The problem:
Given a process, limit the resources it and its child-processes can use. I.e. set CPU time and virtual memory quotas. When the process group exceeds one of the limits, terminate it, otherwise print the amount of CPU time and virtual memory it has used.
The concrete use case:
Basically I must execute a couple of binaries, which expect input from a file, but I must ensure that their execution process is strictly limited. For example the binary must not allocate more than 256 MB of memory and it should run for less than 0.5 seconds. Hope this makes sense now. However I need information about the amount of memory and CPU it has used.
What I have tried:
- For a couple of days I have been dealing with this perl script, which is the best solution I have found so far. Unfortunately its memory is buggy and it is not very precise. Also there is an official author post about this script here.
- I have tried using both
/usr/bin/timeoutandtimeoutLinux tools, which of course help me with the CPU time quota, but not with the termination of the process due to violation of the virtual memory limit. - Using
ulimitwas attempted as well but as I have said earlier I need not only limitation but feedback for the resource consumption too.
The Question:
What can solve this issue? .
Edit 1: Added information for the concrete use case for better understanding of the problem.
process memory monitoring cpu cpu-usage
edited 3 hours ago
Rui F Ribeiro
41.9k1483142
asked Jul 15 '15 at 10:44
0x4500x450
1208
The problem:
Given a process, limit the resources it and its child-processes can use. I.e. set CPU time and virtual memory quotas. When the process group exceeds one of the limits, terminate it, otherwise print the amount of CPU time and virtual memory it has used.
The concrete use case:
Basically I must execute a couple of binaries, which expect input from a file, but I must ensure that their execution process is strictly limited. For example the binary must not allocate more than 256 MB of memory and it should run for less than 0.5 seconds. Hope this makes sense now. However I need information about the amount of memory and CPU it has used.
What I have tried:
- For a couple of days I have been dealing with this perl script, which is the best solution I have found so far. Unfortunately its memory is buggy and it is not very precise. Also there is an official author post about this script here.
- I have tried using both
/usr/bin/timeoutandtimeoutLinux tools, which of course help me with the CPU time quota, but not with the termination of the process due to violation of the virtual memory limit. - Using
ulimitwas attempted as well but as I have said earlier I need not only limitation but feedback for the resource consumption too.
The Question:
What can solve this issue? .
Edit 1: Added information for the concrete use case for better understanding of the problem.
process memory monitoring cpu cpu-usage
process memory monitoring cpu cpu-usage
edited 3 hours ago
Rui F Ribeiro
41.9k1483142
asked Jul 15 '15 at 10:44
0x4500x450
1208
edited 3 hours ago
Rui F Ribeiro
41.9k1483142
asked Jul 15 '15 at 10:44
0x4500x450
1208
edited 3 hours ago
Rui F Ribeiro
41.9k1483142
edited 3 hours ago
Rui F Ribeiro
41.9k1483142
edited 3 hours ago
Rui F Ribeiro
41.9k1483142
41.9k1483142
asked Jul 15 '15 at 10:44
0x4500x450
1208
asked Jul 15 '15 at 10:44
0x4500x450
1208
asked Jul 15 '15 at 10:44
0x4500x450
1208
1208
Your question is too broad IMHO. Could you motivate it more and give more details. So please edit your question to improve it. What is the software you have trouble with? Do you have its source code?
– Basile Starynkevitch
Jul 15 '15 at 11:15
Basically I must execute a couple of binaries, which expect input from a file, but I must ensure that their execution process is strictly limited. For example the binary must not allocate more than 256 MB of memory and it should run for less than 0.5 seconds. Hope this makes sense now. However I need information about the amount of memory and CPU it used.
– 0x450
Jul 15 '15 at 11:24
Please edit your question instead of commenting it. BTW, my answer fits (except thatsetrlimitcan limit CPU time only for an integral number of seconds)
– Basile Starynkevitch
Jul 15 '15 at 11:25
Can you raise the 0.5 CPU second limit to 1 second? If yes, usesetrlimit
– Basile Starynkevitch
Jul 15 '15 at 11:45
Are you able to code in C or any programming language giving access to Linux low level syscalls?
– Basile Starynkevitch
Jul 15 '15 at 11:49
|
show 5 more comments
Your question is too broad IMHO. Could you motivate it more and give more details. So please edit your question to improve it. What is the software you have trouble with? Do you have its source code?
– Basile Starynkevitch
Jul 15 '15 at 11:15
Basically I must execute a couple of binaries, which expect input from a file, but I must ensure that their execution process is strictly limited. For example the binary must not allocate more than 256 MB of memory and it should run for less than 0.5 seconds. Hope this makes sense now. However I need information about the amount of memory and CPU it used.
– 0x450
Jul 15 '15 at 11:24
Please edit your question instead of commenting it. BTW, my answer fits (except thatsetrlimitcan limit CPU time only for an integral number of seconds)
– Basile Starynkevitch
Jul 15 '15 at 11:25
Can you raise the 0.5 CPU second limit to 1 second? If yes, usesetrlimit
– Basile Starynkevitch
Jul 15 '15 at 11:45
Are you able to code in C or any programming language giving access to Linux low level syscalls?
– Basile Starynkevitch
Jul 15 '15 at 11:49
Your question is too broad IMHO. Could you motivate it more and give more details. So please edit your question to improve it. What is the software you have trouble with? Do you have its source code?
– Basile Starynkevitch
Jul 15 '15 at 11:15
Your question is too broad IMHO. Could you motivate it more and give more details. So please edit your question to improve it. What is the software you have trouble with? Do you have its source code?
– Basile Starynkevitch
Jul 15 '15 at 11:15
Basically I must execute a couple of binaries, which expect input from a file, but I must ensure that their execution process is strictly limited. For example the binary must not allocate more than 256 MB of memory and it should run for less than 0.5 seconds. Hope this makes sense now. However I need information about the amount of memory and CPU it used.
– 0x450
Jul 15 '15 at 11:24
Basically I must execute a couple of binaries, which expect input from a file, but I must ensure that their execution process is strictly limited. For example the binary must not allocate more than 256 MB of memory and it should run for less than 0.5 seconds. Hope this makes sense now. However I need information about the amount of memory and CPU it used.
– 0x450
Jul 15 '15 at 11:24
Please edit your question instead of commenting it. BTW, my answer fits (except that
setrlimit can limit CPU time only for an integral number of seconds)– Basile Starynkevitch
Jul 15 '15 at 11:25
Please edit your question instead of commenting it. BTW, my answer fits (except that
setrlimit can limit CPU time only for an integral number of seconds)– Basile Starynkevitch
Jul 15 '15 at 11:25
Can you raise the 0.5 CPU second limit to 1 second? If yes, use
setrlimit– Basile Starynkevitch
Jul 15 '15 at 11:45
Can you raise the 0.5 CPU second limit to 1 second? If yes, use
setrlimit– Basile Starynkevitch
Jul 15 '15 at 11:45
Are you able to code in C or any programming language giving access to Linux low level syscalls?
– Basile Starynkevitch
Jul 15 '15 at 11:49
Are you able to code in C or any programming language giving access to Linux low level syscalls?
– Basile Starynkevitch
Jul 15 '15 at 11:49
|
show 5 more comments
1 Answer
1
active
oldest
votes
The setrlimit(2) syscall is relevant to limit resources (CPU time -integral number of seconds, so at least 1 sec- with RLIMIT_CPU, file size with RLIMIT_FSIZE, address space with RLIMIT_AS, etc...). You could also set up disk quotas. The wait4(2) syscall tells you -and gives feedback- about some resource usage. And proc(5) tells you a lot more, and also getrusage(2) (you might code some monitor which periodically stops the entire process group using SIGSTOP, call getrusage or query /proc/$PID/, then send SIGCONT -to continue- or SIGTERM -to terminate- to that process group).
The valgrind tool is very useful on Linux to help finding memory leaks. And strace(1) should be helpful too.
If you can recompile the faulty software, you could consider -fsanitize=address and -fsanitize=undefined and other -fsanitize=... options to a recent version of the GCC compiler.
Perhaps you have some batch processing. Look for batch monitors, or simply code your own thing in C, Python, Ocaml, Perl, .... (which forks the command, and loop on monitoring it...). Maybe you want some process accounting (see acct(5) & sa(8)...)
Notice that "amount of memory used" (a program generally allocates with mmap & releases memory with munmap to the kernel while running) and "CPU time" (see time(7), think of multi-threaded programs ...) are very fuzzy concepts.
See also PAM and configure things under /etc/security/ ; perhaps inotify(7) might also be helpful (but probably not).
Read also Advanced Linux Programming and syscalls(2)
answered Jul 15 '15 at 11:11
Basile StarynkevitchBasile Starynkevitch
8,1512041
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f216149%2fresource-cpu-time-and-memory-limitation-and-termination-of-a-process-upon-viol%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
The setrlimit(2) syscall is relevant to limit resources (CPU time -integral number of seconds, so at least 1 sec- with RLIMIT_CPU, file size with RLIMIT_FSIZE, address space with RLIMIT_AS, etc...). You could also set up disk quotas. The wait4(2) syscall tells you -and gives feedback- about some resource usage. And proc(5) tells you a lot more, and also getrusage(2) (you might code some monitor which periodically stops the entire process group using SIGSTOP, call getrusage or query /proc/$PID/, then send SIGCONT -to continue- or SIGTERM -to terminate- to that process group).
The valgrind tool is very useful on Linux to help finding memory leaks. And strace(1) should be helpful too.
If you can recompile the faulty software, you could consider -fsanitize=address and -fsanitize=undefined and other -fsanitize=... options to a recent version of the GCC compiler.
Perhaps you have some batch processing. Look for batch monitors, or simply code your own thing in C, Python, Ocaml, Perl, .... (which forks the command, and loop on monitoring it...). Maybe you want some process accounting (see acct(5) & sa(8)...)
Notice that "amount of memory used" (a program generally allocates with mmap & releases memory with munmap to the kernel while running) and "CPU time" (see time(7), think of multi-threaded programs ...) are very fuzzy concepts.
See also PAM and configure things under /etc/security/ ; perhaps inotify(7) might also be helpful (but probably not).
Read also Advanced Linux Programming and syscalls(2)
answered Jul 15 '15 at 11:11
Basile StarynkevitchBasile Starynkevitch
8,1512041
add a comment |
The setrlimit(2) syscall is relevant to limit resources (CPU time -integral number of seconds, so at least 1 sec- with RLIMIT_CPU, file size with RLIMIT_FSIZE, address space with RLIMIT_AS, etc...). You could also set up disk quotas. The wait4(2) syscall tells you -and gives feedback- about some resource usage. And proc(5) tells you a lot more, and also getrusage(2) (you might code some monitor which periodically stops the entire process group using SIGSTOP, call getrusage or query /proc/$PID/, then send SIGCONT -to continue- or SIGTERM -to terminate- to that process group).
The valgrind tool is very useful on Linux to help finding memory leaks. And strace(1) should be helpful too.
If you can recompile the faulty software, you could consider -fsanitize=address and -fsanitize=undefined and other -fsanitize=... options to a recent version of the GCC compiler.
Perhaps you have some batch processing. Look for batch monitors, or simply code your own thing in C, Python, Ocaml, Perl, .... (which forks the command, and loop on monitoring it...). Maybe you want some process accounting (see acct(5) & sa(8)...)
Notice that "amount of memory used" (a program generally allocates with mmap & releases memory with munmap to the kernel while running) and "CPU time" (see time(7), think of multi-threaded programs ...) are very fuzzy concepts.
See also PAM and configure things under /etc/security/ ; perhaps inotify(7) might also be helpful (but probably not).
Read also Advanced Linux Programming and syscalls(2)
answered Jul 15 '15 at 11:11
Basile StarynkevitchBasile Starynkevitch
8,1512041
add a comment |
The setrlimit(2) syscall is relevant to limit resources (CPU time -integral number of seconds, so at least 1 sec- with RLIMIT_CPU, file size with RLIMIT_FSIZE, address space with RLIMIT_AS, etc...). You could also set up disk quotas. The wait4(2) syscall tells you -and gives feedback- about some resource usage. And proc(5) tells you a lot more, and also getrusage(2) (you might code some monitor which periodically stops the entire process group using SIGSTOP, call getrusage or query /proc/$PID/, then send SIGCONT -to continue- or SIGTERM -to terminate- to that process group).
The valgrind tool is very useful on Linux to help finding memory leaks. And strace(1) should be helpful too.
If you can recompile the faulty software, you could consider -fsanitize=address and -fsanitize=undefined and other -fsanitize=... options to a recent version of the GCC compiler.
Perhaps you have some batch processing. Look for batch monitors, or simply code your own thing in C, Python, Ocaml, Perl, .... (which forks the command, and loop on monitoring it...). Maybe you want some process accounting (see acct(5) & sa(8)...)
Notice that "amount of memory used" (a program generally allocates with mmap & releases memory with munmap to the kernel while running) and "CPU time" (see time(7), think of multi-threaded programs ...) are very fuzzy concepts.
See also PAM and configure things under /etc/security/ ; perhaps inotify(7) might also be helpful (but probably not).
Read also Advanced Linux Programming and syscalls(2)
answered Jul 15 '15 at 11:11
Basile StarynkevitchBasile Starynkevitch
8,1512041
The setrlimit(2) syscall is relevant to limit resources (CPU time -integral number of seconds, so at least 1 sec- with RLIMIT_CPU, file size with RLIMIT_FSIZE, address space with RLIMIT_AS, etc...). You could also set up disk quotas. The wait4(2) syscall tells you -and gives feedback- about some resource usage. And proc(5) tells you a lot more, and also getrusage(2) (you might code some monitor which periodically stops the entire process group using SIGSTOP, call getrusage or query /proc/$PID/, then send SIGCONT -to continue- or SIGTERM -to terminate- to that process group).
The valgrind tool is very useful on Linux to help finding memory leaks. And strace(1) should be helpful too.
If you can recompile the faulty software, you could consider -fsanitize=address and -fsanitize=undefined and other -fsanitize=... options to a recent version of the GCC compiler.
Perhaps you have some batch processing. Look for batch monitors, or simply code your own thing in C, Python, Ocaml, Perl, .... (which forks the command, and loop on monitoring it...). Maybe you want some process accounting (see acct(5) & sa(8)...)
Notice that "amount of memory used" (a program generally allocates with mmap & releases memory with munmap to the kernel while running) and "CPU time" (see time(7), think of multi-threaded programs ...) are very fuzzy concepts.
See also PAM and configure things under /etc/security/ ; perhaps inotify(7) might also be helpful (but probably not).
Read also Advanced Linux Programming and syscalls(2)
answered Jul 15 '15 at 11:11
Basile StarynkevitchBasile Starynkevitch
8,1512041
edited Jul 15 '15 at 11:48
answered Jul 15 '15 at 11:11
Basile StarynkevitchBasile Starynkevitch
8,1512041
answered Jul 15 '15 at 11:11
Basile StarynkevitchBasile Starynkevitch
8,1512041
answered Jul 15 '15 at 11:11
Basile StarynkevitchBasile Starynkevitch
8,1512041
8,1512041
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f216149%2fresource-cpu-time-and-memory-limitation-and-termination-of-a-process-upon-viol%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Your question is too broad IMHO. Could you motivate it more and give more details. So please edit your question to improve it. What is the software you have trouble with? Do you have its source code?
– Basile Starynkevitch
Jul 15 '15 at 11:15
Basically I must execute a couple of binaries, which expect input from a file, but I must ensure that their execution process is strictly limited. For example the binary must not allocate more than 256 MB of memory and it should run for less than 0.5 seconds. Hope this makes sense now. However I need information about the amount of memory and CPU it used.
– 0x450
Jul 15 '15 at 11:24
Please edit your question instead of commenting it. BTW, my answer fits (except that
setrlimitcan limit CPU time only for an integral number of seconds)– Basile Starynkevitch
Jul 15 '15 at 11:25
Can you raise the 0.5 CPU second limit to 1 second? If yes, use
setrlimit– Basile Starynkevitch
Jul 15 '15 at 11:45
Are you able to code in C or any programming language giving access to Linux low level syscalls?
– Basile Starynkevitch
Jul 15 '15 at 11:49