How to filter logs between a time range
Here's my log format(simplified for demonstrating)
2018-04-12 14:43:00.000 ERROR hello
2018-04-12 14:44:01.000 ERROR world
2018-04-12 14:44:03.000 INFO this is a multi-line log
NOTICE THIS LINE, this line is also part of the log
2018-04-12 14:46:00.000 INFO foo
So how to filter the log of [2018-04-12 14:44:00.000, 2018-04-12 14:45:00.000) to produce the following output?
2018-04-12 14:44:01.000 ERROR world
2018-04-12 14:44:03.000 INFO this is a multi-line log
NOTICE THIS LINE, this line is also part of the log
text-processing awk sed grep
asked Apr 13 '18 at 2:00
aLeXaLeX
1011
add a comment |
Here's my log format(simplified for demonstrating)
2018-04-12 14:43:00.000 ERROR hello
2018-04-12 14:44:01.000 ERROR world
2018-04-12 14:44:03.000 INFO this is a multi-line log
NOTICE THIS LINE, this line is also part of the log
2018-04-12 14:46:00.000 INFO foo
So how to filter the log of [2018-04-12 14:44:00.000, 2018-04-12 14:45:00.000) to produce the following output?
2018-04-12 14:44:01.000 ERROR world
2018-04-12 14:44:03.000 INFO this is a multi-line log
NOTICE THIS LINE, this line is also part of the log
text-processing awk sed grep
asked Apr 13 '18 at 2:00
aLeXaLeX
1011
So you're trying to get the log between a minute14:44:00.000and14:45:00.000. Then I guess between that time, there are countless number of lines that will be produce right?`
– WashichawbachaW
Apr 13 '18 at 7:01
@WashichawbachaW yes exactly
– aLeX
Apr 13 '18 at 8:45
add a comment |
Here's my log format(simplified for demonstrating)
2018-04-12 14:43:00.000 ERROR hello
2018-04-12 14:44:01.000 ERROR world
2018-04-12 14:44:03.000 INFO this is a multi-line log
NOTICE THIS LINE, this line is also part of the log
2018-04-12 14:46:00.000 INFO foo
So how to filter the log of [2018-04-12 14:44:00.000, 2018-04-12 14:45:00.000) to produce the following output?
2018-04-12 14:44:01.000 ERROR world
2018-04-12 14:44:03.000 INFO this is a multi-line log
NOTICE THIS LINE, this line is also part of the log
text-processing awk sed grep
asked Apr 13 '18 at 2:00
aLeXaLeX
1011
Here's my log format(simplified for demonstrating)
2018-04-12 14:43:00.000 ERROR hello
2018-04-12 14:44:01.000 ERROR world
2018-04-12 14:44:03.000 INFO this is a multi-line log
NOTICE THIS LINE, this line is also part of the log
2018-04-12 14:46:00.000 INFO foo
So how to filter the log of [2018-04-12 14:44:00.000, 2018-04-12 14:45:00.000) to produce the following output?
2018-04-12 14:44:01.000 ERROR world
2018-04-12 14:44:03.000 INFO this is a multi-line log
NOTICE THIS LINE, this line is also part of the log
text-processing awk sed grep
text-processing awk sed grep
asked Apr 13 '18 at 2:00
aLeXaLeX
1011
asked Apr 13 '18 at 2:00
aLeXaLeX
1011
asked Apr 13 '18 at 2:00
aLeXaLeX
1011
asked Apr 13 '18 at 2:00
aLeXaLeX
1011
asked Apr 13 '18 at 2:00
aLeXaLeX
1011
1011
So you're trying to get the log between a minute14:44:00.000and14:45:00.000. Then I guess between that time, there are countless number of lines that will be produce right?`
– WashichawbachaW
Apr 13 '18 at 7:01
@WashichawbachaW yes exactly
– aLeX
Apr 13 '18 at 8:45
add a comment |
So you're trying to get the log between a minute14:44:00.000and14:45:00.000. Then I guess between that time, there are countless number of lines that will be produce right?`
– WashichawbachaW
Apr 13 '18 at 7:01
@WashichawbachaW yes exactly
– aLeX
Apr 13 '18 at 8:45
So you're trying to get the log between a minute
14:44:00.000 and 14:45:00.000. Then I guess between that time, there are countless number of lines that will be produce right?`– WashichawbachaW
Apr 13 '18 at 7:01
So you're trying to get the log between a minute
14:44:00.000 and 14:45:00.000. Then I guess between that time, there are countless number of lines that will be produce right?`– WashichawbachaW
Apr 13 '18 at 7:01
@WashichawbachaW yes exactly
– aLeX
Apr 13 '18 at 8:45
@WashichawbachaW yes exactly
– aLeX
Apr 13 '18 at 8:45
add a comment |
2 Answers
2
active
oldest
votes
If you just want particular lines between a certain time then awk will work. To give a slight tutorial
To start with and find out which lines you want:
cat -n logfile
That will show the contents of the file with the line numbers.
To print out the line numbers that you want:
awk 'NR==2,NR==4' logfile
That prints out the range between lines 2 and 4.
If you want to print out two ore more lines or a range of lines that aren't consecutive in case you want that then you can separate them with either || or ;
awk 'NR==5,NR==10;NR==15,NR==20' logfile
Moving on to printing the lines between a certain time range, combine the above with grep egrep:
egrep "2018-04-12 14:44:01.000|2018-04-12 14:46:00.000" logfile | awk NR==5,NR==10
egrep allows multiple strings to be returned. The | symbol separates each string. That will print the lines with the start and end of the time range (I changed the end to a later time to include more lines) along with their line number. You can then use awk to print the range between and including the two lines.
You can take all of this as an example and modify it to suit your needs for your log files and what you want to print out according to the times.
answered Apr 13 '18 at 2:20
Nasir RileyNasir Riley
2,406239
add a comment |
You can do this with sed
sed -n '/2018-04-12 14:44:00.000/,/2018-04-12 14:45:00.000/p' log_file
It is worth noting this will only match the first instance, it just uses the dates as a delimiter to print between.
Similar thing can be achieved with awk:
awk '/^2018-04-12 14:44:00.000.*/,/2018-04-12 14:45:00.000.*/' log_file
answered Apr 13 '18 at 6:45
alphaalpha
1,232317
@aLeX this is not a correct answer. What it does is print the line matching the pattern you inputted. If your log file has no matching line, it won't print anything even though theirs a line with log time of 14:44 - 14:45.
– WashichawbachaW
Apr 17 '18 at 1:35
@WashichawbachaW thanks for your mention. I accepted this because of the usage of the comma(,) insedandawk.
– aLeX
Apr 17 '18 at 7:37
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f437415%2fhow-to-filter-logs-between-a-time-range%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
If you just want particular lines between a certain time then awk will work. To give a slight tutorial
To start with and find out which lines you want:
cat -n logfile
That will show the contents of the file with the line numbers.
To print out the line numbers that you want:
awk 'NR==2,NR==4' logfile
That prints out the range between lines 2 and 4.
If you want to print out two ore more lines or a range of lines that aren't consecutive in case you want that then you can separate them with either || or ;
awk 'NR==5,NR==10;NR==15,NR==20' logfile
Moving on to printing the lines between a certain time range, combine the above with grep egrep:
egrep "2018-04-12 14:44:01.000|2018-04-12 14:46:00.000" logfile | awk NR==5,NR==10
egrep allows multiple strings to be returned. The | symbol separates each string. That will print the lines with the start and end of the time range (I changed the end to a later time to include more lines) along with their line number. You can then use awk to print the range between and including the two lines.
You can take all of this as an example and modify it to suit your needs for your log files and what you want to print out according to the times.
answered Apr 13 '18 at 2:20
Nasir RileyNasir Riley
2,406239
add a comment |
If you just want particular lines between a certain time then awk will work. To give a slight tutorial
To start with and find out which lines you want:
cat -n logfile
That will show the contents of the file with the line numbers.
To print out the line numbers that you want:
awk 'NR==2,NR==4' logfile
That prints out the range between lines 2 and 4.
If you want to print out two ore more lines or a range of lines that aren't consecutive in case you want that then you can separate them with either || or ;
awk 'NR==5,NR==10;NR==15,NR==20' logfile
Moving on to printing the lines between a certain time range, combine the above with grep egrep:
egrep "2018-04-12 14:44:01.000|2018-04-12 14:46:00.000" logfile | awk NR==5,NR==10
egrep allows multiple strings to be returned. The | symbol separates each string. That will print the lines with the start and end of the time range (I changed the end to a later time to include more lines) along with their line number. You can then use awk to print the range between and including the two lines.
You can take all of this as an example and modify it to suit your needs for your log files and what you want to print out according to the times.
answered Apr 13 '18 at 2:20
Nasir RileyNasir Riley
2,406239
add a comment |
If you just want particular lines between a certain time then awk will work. To give a slight tutorial
To start with and find out which lines you want:
cat -n logfile
That will show the contents of the file with the line numbers.
To print out the line numbers that you want:
awk 'NR==2,NR==4' logfile
That prints out the range between lines 2 and 4.
If you want to print out two ore more lines or a range of lines that aren't consecutive in case you want that then you can separate them with either || or ;
awk 'NR==5,NR==10;NR==15,NR==20' logfile
Moving on to printing the lines between a certain time range, combine the above with grep egrep:
egrep "2018-04-12 14:44:01.000|2018-04-12 14:46:00.000" logfile | awk NR==5,NR==10
egrep allows multiple strings to be returned. The | symbol separates each string. That will print the lines with the start and end of the time range (I changed the end to a later time to include more lines) along with their line number. You can then use awk to print the range between and including the two lines.
You can take all of this as an example and modify it to suit your needs for your log files and what you want to print out according to the times.
answered Apr 13 '18 at 2:20
Nasir RileyNasir Riley
2,406239
If you just want particular lines between a certain time then awk will work. To give a slight tutorial
To start with and find out which lines you want:
cat -n logfile
That will show the contents of the file with the line numbers.
To print out the line numbers that you want:
awk 'NR==2,NR==4' logfile
That prints out the range between lines 2 and 4.
If you want to print out two ore more lines or a range of lines that aren't consecutive in case you want that then you can separate them with either || or ;
awk 'NR==5,NR==10;NR==15,NR==20' logfile
Moving on to printing the lines between a certain time range, combine the above with grep egrep:
egrep "2018-04-12 14:44:01.000|2018-04-12 14:46:00.000" logfile | awk NR==5,NR==10
egrep allows multiple strings to be returned. The | symbol separates each string. That will print the lines with the start and end of the time range (I changed the end to a later time to include more lines) along with their line number. You can then use awk to print the range between and including the two lines.
You can take all of this as an example and modify it to suit your needs for your log files and what you want to print out according to the times.
answered Apr 13 '18 at 2:20
Nasir RileyNasir Riley
2,406239
edited 26 mins ago
answered Apr 13 '18 at 2:20
Nasir RileyNasir Riley
2,406239
answered Apr 13 '18 at 2:20
Nasir RileyNasir Riley
2,406239
answered Apr 13 '18 at 2:20
Nasir RileyNasir Riley
2,406239
2,406239
add a comment |
add a comment |
You can do this with sed
sed -n '/2018-04-12 14:44:00.000/,/2018-04-12 14:45:00.000/p' log_file
It is worth noting this will only match the first instance, it just uses the dates as a delimiter to print between.
Similar thing can be achieved with awk:
awk '/^2018-04-12 14:44:00.000.*/,/2018-04-12 14:45:00.000.*/' log_file
answered Apr 13 '18 at 6:45
alphaalpha
1,232317
@aLeX this is not a correct answer. What it does is print the line matching the pattern you inputted. If your log file has no matching line, it won't print anything even though theirs a line with log time of 14:44 - 14:45.
– WashichawbachaW
Apr 17 '18 at 1:35
@WashichawbachaW thanks for your mention. I accepted this because of the usage of the comma(,) insedandawk.
– aLeX
Apr 17 '18 at 7:37
add a comment |
You can do this with sed
sed -n '/2018-04-12 14:44:00.000/,/2018-04-12 14:45:00.000/p' log_file
It is worth noting this will only match the first instance, it just uses the dates as a delimiter to print between.
Similar thing can be achieved with awk:
awk '/^2018-04-12 14:44:00.000.*/,/2018-04-12 14:45:00.000.*/' log_file
answered Apr 13 '18 at 6:45
alphaalpha
1,232317
@aLeX this is not a correct answer. What it does is print the line matching the pattern you inputted. If your log file has no matching line, it won't print anything even though theirs a line with log time of 14:44 - 14:45.
– WashichawbachaW
Apr 17 '18 at 1:35
@WashichawbachaW thanks for your mention. I accepted this because of the usage of the comma(,) insedandawk.
– aLeX
Apr 17 '18 at 7:37
add a comment |
You can do this with sed
sed -n '/2018-04-12 14:44:00.000/,/2018-04-12 14:45:00.000/p' log_file
It is worth noting this will only match the first instance, it just uses the dates as a delimiter to print between.
Similar thing can be achieved with awk:
awk '/^2018-04-12 14:44:00.000.*/,/2018-04-12 14:45:00.000.*/' log_file
answered Apr 13 '18 at 6:45
alphaalpha
1,232317
You can do this with sed
sed -n '/2018-04-12 14:44:00.000/,/2018-04-12 14:45:00.000/p' log_file
It is worth noting this will only match the first instance, it just uses the dates as a delimiter to print between.
Similar thing can be achieved with awk:
awk '/^2018-04-12 14:44:00.000.*/,/2018-04-12 14:45:00.000.*/' log_file
answered Apr 13 '18 at 6:45
alphaalpha
1,232317
answered Apr 13 '18 at 6:45
alphaalpha
1,232317
answered Apr 13 '18 at 6:45
alphaalpha
1,232317
answered Apr 13 '18 at 6:45
alphaalpha
1,232317
1,232317
@aLeX this is not a correct answer. What it does is print the line matching the pattern you inputted. If your log file has no matching line, it won't print anything even though theirs a line with log time of 14:44 - 14:45.
– WashichawbachaW
Apr 17 '18 at 1:35
@WashichawbachaW thanks for your mention. I accepted this because of the usage of the comma(,) insedandawk.
– aLeX
Apr 17 '18 at 7:37
add a comment |
@aLeX this is not a correct answer. What it does is print the line matching the pattern you inputted. If your log file has no matching line, it won't print anything even though theirs a line with log time of 14:44 - 14:45.
– WashichawbachaW
Apr 17 '18 at 1:35
@WashichawbachaW thanks for your mention. I accepted this because of the usage of the comma(,) insedandawk.
– aLeX
Apr 17 '18 at 7:37
@aLeX this is not a correct answer. What it does is print the line matching the pattern you inputted. If your log file has no matching line, it won't print anything even though theirs a line with log time of 14:44 - 14:45.
– WashichawbachaW
Apr 17 '18 at 1:35
@aLeX this is not a correct answer. What it does is print the line matching the pattern you inputted. If your log file has no matching line, it won't print anything even though theirs a line with log time of 14:44 - 14:45.
– WashichawbachaW
Apr 17 '18 at 1:35
@WashichawbachaW thanks for your mention. I accepted this because of the usage of the comma(
,) in sed and awk.– aLeX
Apr 17 '18 at 7:37
@WashichawbachaW thanks for your mention. I accepted this because of the usage of the comma(
,) in sed and awk.– aLeX
Apr 17 '18 at 7:37
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f437415%2fhow-to-filter-logs-between-a-time-range%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
So you're trying to get the log between a minute
14:44:00.000and14:45:00.000. Then I guess between that time, there are countless number of lines that will be produce right?`– WashichawbachaW
Apr 13 '18 at 7:01
@WashichawbachaW yes exactly
– aLeX
Apr 13 '18 at 8:45