mirror of
https://codeberg.org/icewind/logsmash.git
synced 2026-06-03 18:14:11 +02:00
formatting
This commit is contained in:
parent
085da1a195
commit
2197181ab4
7 changed files with 132 additions and 126 deletions
70
README.md
70
README.md
|
|
@ -6,8 +6,9 @@ Analysis tool for Nextcloud logs files
|
||||||
|
|
||||||
## Quickstart
|
## Quickstart
|
||||||
|
|
||||||
Download the binary from the [releases page](https://github.com/icewind1991/logsmash/releases), place it somewhere in
|
Download the binary from the
|
||||||
your `$PATH` and make it executable.
|
[releases page](https://github.com/icewind1991/logsmash/releases), place it
|
||||||
|
somewhere in your `$PATH` and make it executable.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
logsmash ./logfile.log
|
logsmash ./logfile.log
|
||||||
|
|
@ -15,34 +16,37 @@ logsmash ./logfile.log
|
||||||
|
|
||||||
## Log files
|
## Log files
|
||||||
|
|
||||||
Logsmash supports both loading plain log files, compressed log files (`.gz`, `.bz2`, `.xz` or `.zst`), or archives
|
Logsmash supports both loading plain log files, compressed log files (`.gz`,
|
||||||
containing log files (`.zip` or (compressed) `.tar`).
|
`.bz2`, `.xz` or `.zst`), or archives containing log files (`.zip` or
|
||||||
|
(compressed) `.tar`).
|
||||||
|
|
||||||
### Date formats
|
### Date formats
|
||||||
|
|
||||||
Since not all log files use the same date format, logsmash tries to parse each data with a number of different log
|
Since not all log files use the same date format, logsmash tries to parse each
|
||||||
formats.
|
data with a number of different log formats.
|
||||||
|
|
||||||
If the log file you're opening is using an unsupported log format, you can specify a custom date format with the
|
If the log file you're opening is using an unsupported log format, you can
|
||||||
`--date-format` option.
|
specify a custom date format with the `--date-format` option.
|
||||||
|
|
||||||
The data format can either be in [the strftime format](https://man7.org/linux/man-pages/man3/strftime.3.html) or
|
The data format can either be in
|
||||||
in [the time crate format (version 2)](https://time-rs.github.io/book/api/format-description.html).
|
[the strftime format](https://man7.org/linux/man-pages/man3/strftime.3.html) or
|
||||||
|
in
|
||||||
|
[the time crate format (version 2)](https://time-rs.github.io/book/api/format-description.html).
|
||||||
|
|
||||||
For example: `[day].[month].[year] - [hour]:[minute]:[second]`.
|
For example: `[day].[month].[year] - [hour]:[minute]:[second]`.
|
||||||
|
|
||||||
## Log sources
|
## Log sources
|
||||||
|
|
||||||
Logsmash is built around matching log line to their source, either a call to a logging function or an exception being
|
Logsmash is built around matching log line to their source, either a call to a
|
||||||
thrown.
|
logging function or an exception being thrown. Many log lines do not include
|
||||||
Many log lines do not include information about their source to logsmash attempts to find the source of the log by
|
information about their source to logsmash attempts to find the source of the
|
||||||
comparing
|
log by comparing the logged message against a list of known log sources included
|
||||||
the logged message against a list of known log sources included in the binary.
|
in the binary.
|
||||||
|
|
||||||
Since multiple log sources can create similar log lines, some lines will match to multiple log sources, in those cases
|
Since multiple log sources can create similar log lines, some lines will match
|
||||||
logsmash
|
to multiple log sources, in those cases logsmash will show all the matched
|
||||||
will show all the matched sources as a single item.
|
sources as a single item. Log items that cannot be matched to any known source
|
||||||
Log items that cannot be matched to any known source are grouped together
|
are grouped together
|
||||||
|
|
||||||
## UI overview
|
## UI overview
|
||||||
|
|
||||||
|
|
@ -50,14 +54,15 @@ Logsmash contains the following UI pages:
|
||||||
|
|
||||||
### Overview
|
### Overview
|
||||||
|
|
||||||
Shows a list of all matched sources with their source location, the number of log items matched to the source, and a
|
Shows a list of all matched sources with their source location, the number of
|
||||||
graph of log occurrence over time.
|
log items matched to the source, and a graph of log occurrence over time.
|
||||||
|
|
||||||
Selecting an item will bring you to the source overview for the log source.
|
Selecting an item will bring you to the source overview for the log source.
|
||||||
|
|
||||||
### Source overview
|
### Source overview
|
||||||
|
|
||||||
Shows a list of distinct log lines (lines with the same message, level and context).
|
Shows a list of distinct log lines (lines with the same message, level and
|
||||||
|
context).
|
||||||
|
|
||||||
Selecting an item will bring you to the log list
|
Selecting an item will bring you to the log list
|
||||||
|
|
||||||
|
|
@ -69,22 +74,23 @@ Selecting an item will bring you to the log page
|
||||||
|
|
||||||
### Log page
|
### Log page
|
||||||
|
|
||||||
Shows the message and accompanying metadata for the log line. For log lines with exceptions, the exception backtrace is
|
Shows the message and accompanying metadata for the log line. For log lines with
|
||||||
shown.
|
exceptions, the exception backtrace is shown.
|
||||||
|
|
||||||
### Log occurrence graph
|
### Log occurrence graph
|
||||||
|
|
||||||
Lists that contain grouped log lines come with a graph that show how often the group of log lines occurred over time as
|
Lists that contain grouped log lines come with a graph that show how often the
|
||||||
a histogram.
|
group of log lines occurred over time as a histogram. Each line in the list will
|
||||||
Each line in the list will show a small version of the graph, and a larger version of the graph for the selected item is
|
show a small version of the graph, and a larger version of the graph for the
|
||||||
shown above the list.
|
selected item is shown above the list.
|
||||||
|
|
||||||
## Keyboard controls
|
## Keyboard controls
|
||||||
|
|
||||||
- All lists are navigated with the arrow keys, `PgUp`/`PgDown` and `Home`/`End`.
|
- All lists are navigated with the arrow keys, `PgUp`/`PgDown` and `Home`/`End`.
|
||||||
- Items in lists can be selected with `Enter` or the right arrow key.
|
- Items in lists can be selected with `Enter` or the right arrow key.
|
||||||
- You can return to the previous page with `Esc` or the left arrow key.
|
- You can return to the previous page with `Esc` or the left arrow key.
|
||||||
- Single log items can be copied as json with `c` (requires [OSC 52 support](https://github.com/ojroques/vim-oscyank))
|
- Single log items can be copied as json with `c` (requires
|
||||||
|
[OSC 52 support](https://github.com/ojroques/vim-oscyank))
|
||||||
|
|
||||||
## Supported data
|
## Supported data
|
||||||
|
|
||||||
|
|
@ -119,14 +125,16 @@ Currently, the program can match against data from the following sources:
|
||||||
|
|
||||||
## Building
|
## Building
|
||||||
|
|
||||||
The preferred way of building is trough [`nix`](https://nixos.org/download/#download-nix), this ensures that all baked
|
The preferred way of building is trough
|
||||||
|
[`nix`](https://nixos.org/download/#download-nix), this ensures that all baked
|
||||||
data is automatically up-to-date.
|
data is automatically up-to-date.
|
||||||
|
|
||||||
`nix build .#logsmash`
|
`nix build .#logsmash`
|
||||||
|
|
||||||
#### Updating baked data
|
#### Updating baked data
|
||||||
|
|
||||||
Note that this is only needed when building with cargo, building with nix automatically uses the latest data.
|
Note that this is only needed when building with cargo, building with nix
|
||||||
|
automatically uses the latest data.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
rm -r data/src/data
|
rm -r data/src/data
|
||||||
|
|
|
||||||
|
|
@ -10,12 +10,13 @@
|
||||||
inputs.flakelight.follows = "flakelight";
|
inputs.flakelight.follows = "flakelight";
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
outputs = { mill-scale, ... }: mill-scale ./. {
|
outputs = {mill-scale, ...}:
|
||||||
|
mill-scale ./. {
|
||||||
crossTargets = [
|
crossTargets = [
|
||||||
"x86_64-unknown-linux-musl"
|
"x86_64-unknown-linux-musl"
|
||||||
"x86_64-pc-windows-gnu"
|
"x86_64-pc-windows-gnu"
|
||||||
];
|
];
|
||||||
packageOpts = { extracted-logs-rust, ... }: {
|
packageOpts = {extracted-logs-rust, ...}: {
|
||||||
preBuild = ''
|
preBuild = ''
|
||||||
rm -r -f data/src/data
|
rm -r -f data/src/data
|
||||||
cp -r ${extracted-logs-rust} data/src/data
|
cp -r ${extracted-logs-rust} data/src/data
|
||||||
|
|
@ -23,8 +24,8 @@
|
||||||
};
|
};
|
||||||
withOverlays = import ./nix/overlay.nix;
|
withOverlays = import ./nix/overlay.nix;
|
||||||
packages = {
|
packages = {
|
||||||
logsmash = { logsmash }: logsmash;
|
logsmash = {logsmash}: logsmash;
|
||||||
extracted-logs-rust = { extracted-logs-rust }: extracted-logs-rust;
|
extracted-logs-rust = {extracted-logs-rust}: extracted-logs-rust;
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -1,9 +1,8 @@
|
||||||
{ packages
|
{
|
||||||
, lib
|
packages,
|
||||||
, writeTextDir
|
lib,
|
||||||
,
|
writeTextDir,
|
||||||
}:
|
}: let
|
||||||
let
|
|
||||||
inherit (builtins) head attrNames concatStringsSep replaceStrings;
|
inherit (builtins) head attrNames concatStringsSep replaceStrings;
|
||||||
inherit (lib.attrsets) mapAttrsToList;
|
inherit (lib.attrsets) mapAttrsToList;
|
||||||
inherit (lib.lists) flatten last;
|
inherit (lib.lists) flatten last;
|
||||||
|
|
@ -35,4 +34,4 @@ let
|
||||||
}
|
}
|
||||||
'';
|
'';
|
||||||
in
|
in
|
||||||
writeTextDir "mod.rs" code
|
writeTextDir "mod.rs" code
|
||||||
|
|
|
||||||
|
|
@ -1,19 +1,21 @@
|
||||||
{ stdenvNoCC
|
{
|
||||||
, logging-extractor
|
stdenvNoCC,
|
||||||
, fetchzip
|
logging-extractor,
|
||||||
, name
|
fetchzip,
|
||||||
, version
|
name,
|
||||||
, url
|
version,
|
||||||
, major
|
url,
|
||||||
, sha256
|
major,
|
||||||
, mode ? "json"
|
sha256,
|
||||||
,
|
mode ? "json",
|
||||||
}:
|
}: let
|
||||||
let
|
ext =
|
||||||
ext = if mode == "rust" then "rs" else "json";
|
if mode == "rust"
|
||||||
cleanedMajor = builtins.replaceStrings [ "." ] [ "_" ] major;
|
then "rs"
|
||||||
|
else "json";
|
||||||
|
cleanedMajor = builtins.replaceStrings ["."] ["_"] major;
|
||||||
in
|
in
|
||||||
stdenvNoCC.mkDerivation rec {
|
stdenvNoCC.mkDerivation rec {
|
||||||
pname = "extractor-logs-${name}-${cleanedMajor}";
|
pname = "extractor-logs-${name}-${cleanedMajor}";
|
||||||
inherit version;
|
inherit version;
|
||||||
|
|
||||||
|
|
@ -21,7 +23,7 @@ stdenvNoCC.mkDerivation rec {
|
||||||
inherit url sha256;
|
inherit url sha256;
|
||||||
};
|
};
|
||||||
|
|
||||||
nativeBuildInputs = [ logging-extractor ];
|
nativeBuildInputs = [logging-extractor];
|
||||||
|
|
||||||
buildPhase = ''
|
buildPhase = ''
|
||||||
logging-extractor . ${mode} > logs.${ext}
|
logging-extractor . ${mode} > logs.${ext}
|
||||||
|
|
@ -31,4 +33,4 @@ stdenvNoCC.mkDerivation rec {
|
||||||
mkdir -p $out
|
mkdir -p $out
|
||||||
cp logs.* $out/${name}_${cleanedMajor}.${ext}
|
cp logs.* $out/${name}_${cleanedMajor}.${ext}
|
||||||
'';
|
'';
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -1,13 +1,12 @@
|
||||||
{ stdenv
|
{
|
||||||
, rustPlatform
|
stdenv,
|
||||||
, lib
|
rustPlatform,
|
||||||
,
|
lib,
|
||||||
}:
|
}: let
|
||||||
let
|
|
||||||
inherit (lib.sources) sourceByRegex;
|
inherit (lib.sources) sourceByRegex;
|
||||||
src = sourceByRegex ../logging-extractor [ "Cargo.*" "(src|tests|test-data)(/.*)?" ];
|
src = sourceByRegex ../logging-extractor ["Cargo.*" "(src|tests|test-data)(/.*)?"];
|
||||||
in
|
in
|
||||||
rustPlatform.buildRustPackage rec {
|
rustPlatform.buildRustPackage rec {
|
||||||
pname = "logging-extractor";
|
pname = "logging-extractor";
|
||||||
version = "0.1.0";
|
version = "0.1.0";
|
||||||
|
|
||||||
|
|
@ -16,4 +15,4 @@ rustPlatform.buildRustPackage rec {
|
||||||
cargoLock = {
|
cargoLock = {
|
||||||
lockFile = ../logging-extractor/Cargo.lock;
|
lockFile = ../logging-extractor/Cargo.lock;
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -1,20 +1,19 @@
|
||||||
{ stdenv
|
{
|
||||||
, makeRustPlatform
|
stdenv,
|
||||||
, rust-bin
|
makeRustPlatform,
|
||||||
, lib
|
rust-bin,
|
||||||
, extracted-logs-rust
|
lib,
|
||||||
,
|
extracted-logs-rust,
|
||||||
}:
|
}: let
|
||||||
let
|
|
||||||
inherit (lib) sourceByRegex readFile;
|
inherit (lib) sourceByRegex readFile;
|
||||||
src = sourceByRegex ../. [ "Cargo.*" "(src|data)(/.*)?" ];
|
src = sourceByRegex ../. ["Cargo.*" "(src|data)(/.*)?"];
|
||||||
rustPlatform = makeRustPlatform {
|
rustPlatform = makeRustPlatform {
|
||||||
cargo = rust-bin.stable.latest.default;
|
cargo = rust-bin.stable.latest.default;
|
||||||
rustc = rust-bin.stable.latest.default;
|
rustc = rust-bin.stable.latest.default;
|
||||||
};
|
};
|
||||||
version = (fromTOML (readFile ../Cargo.toml)).package.version;
|
version = (fromTOML (readFile ../Cargo.toml)).package.version;
|
||||||
in
|
in
|
||||||
rustPlatform.buildRustPackage rec {
|
rustPlatform.buildRustPackage rec {
|
||||||
pname = "logsmash";
|
pname = "logsmash";
|
||||||
inherit version src;
|
inherit version src;
|
||||||
|
|
||||||
|
|
@ -26,4 +25,4 @@ rustPlatform.buildRustPackage rec {
|
||||||
cargoLock = {
|
cargoLock = {
|
||||||
lockFile = ../Cargo.lock;
|
lockFile = ../Cargo.lock;
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,4 @@
|
||||||
final: prev:
|
final: prev: let
|
||||||
let
|
|
||||||
inherit (builtins) mapAttrs attrValues map tail;
|
inherit (builtins) mapAttrs attrValues map tail;
|
||||||
inherit (prev) symlinkJoin;
|
inherit (prev) symlinkJoin;
|
||||||
inherit (prev.lib) importJSON;
|
inherit (prev.lib) importJSON;
|
||||||
|
|
@ -16,19 +15,18 @@ let
|
||||||
inherit (data) url sha256 version;
|
inherit (data) url sha256 version;
|
||||||
inherit name major mode;
|
inherit name major mode;
|
||||||
}));
|
}));
|
||||||
in
|
in {
|
||||||
{
|
logging-extractor = final.callPackage ./logging-extractor.nix {};
|
||||||
logging-extractor = final.callPackage ./logging-extractor.nix { };
|
|
||||||
extracted-logs-parts = mapAttrs (loggingFor "json") packages;
|
extracted-logs-parts = mapAttrs (loggingFor "json") packages;
|
||||||
extracted-logs-parts-rust = mapAttrs (loggingFor "rust") packages;
|
extracted-logs-parts-rust = mapAttrs (loggingFor "rust") packages;
|
||||||
extracted-logs-rust-mod = final.callPackage ./data-mod.nix { inherit packages; };
|
extracted-logs-rust-mod = final.callPackage ./data-mod.nix {inherit packages;};
|
||||||
extracted-logs = symlinkJoin {
|
extracted-logs = symlinkJoin {
|
||||||
name = "extracted-logs";
|
name = "extracted-logs";
|
||||||
paths = flatten (map attrValues (attrValues final.extracted-logs-parts));
|
paths = flatten (map attrValues (attrValues final.extracted-logs-parts));
|
||||||
};
|
};
|
||||||
extracted-logs-rust = symlinkJoin {
|
extracted-logs-rust = symlinkJoin {
|
||||||
name = "extracted-logs";
|
name = "extracted-logs";
|
||||||
paths = (flatten (map attrValues (attrValues final.extracted-logs-parts-rust))) ++ [ final.extracted-logs-rust-mod ];
|
paths = (flatten (map attrValues (attrValues final.extracted-logs-parts-rust))) ++ [final.extracted-logs-rust-mod];
|
||||||
};
|
};
|
||||||
logsmash = final.callPackage ./logsmash.nix { };
|
logsmash = final.callPackage ./logsmash.nix {};
|
||||||
}
|
}
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue