robin/logsmash
1
0
Fork 0
mirror of https://codeberg.org/icewind/logsmash.git synced 2026-06-03 18:14:11 +02:00
Code Issues Projects Releases 3 Packages Wiki Activity Actions

formatting

Browse source
This commit is contained in:
Robin Appelman 2025-05-26 18:42:12 +02:00
commit 2197181ab4
7 changed files with 132 additions and 126 deletions
Download patch file Download diff file Expand all files Collapse all files

70
README.md
View file

@ -6,8 +6,9 @@ Analysis tool for Nextcloud logs files
## Quickstart
Download the binary from the [releases page](https://github.com/icewind1991/logsmash/releases), place it somewhere in
your `$PATH` and make it executable.
Download the binary from the
[releases page](https://github.com/icewind1991/logsmash/releases), place it
somewhere in your `$PATH` and make it executable.
```bash
logsmash ./logfile.log
@ -15,34 +16,37 @@ logsmash ./logfile.log
## Log files
Logsmash supports both loading plain log files, compressed log files (`.gz`, `.bz2`, `.xz` or `.zst`), or archives
containing log files (`.zip` or (compressed) `.tar`).
Logsmash supports both loading plain log files, compressed log files (`.gz`,
`.bz2`, `.xz` or `.zst`), or archives containing log files (`.zip` or
(compressed) `.tar`).
### Date formats
Since not all log files use the same date format, logsmash tries to parse each data with a number of different log
formats.
Since not all log files use the same date format, logsmash tries to parse each
data with a number of different log formats.
If the log file you're opening is using an unsupported log format, you can specify a custom date format with the
`--date-format` option.
If the log file you're opening is using an unsupported log format, you can
specify a custom date format with the `--date-format` option.
The data format can either be in [the strftime format](https://man7.org/linux/man-pages/man3/strftime.3.html) or
in [the time crate format (version 2)](https://time-rs.github.io/book/api/format-description.html).
The data format can either be in
[the strftime format](https://man7.org/linux/man-pages/man3/strftime.3.html) or
in
[the time crate format (version 2)](https://time-rs.github.io/book/api/format-description.html).
For example: `[day].[month].[year] - [hour]:[minute]:[second]`.
## Log sources
Logsmash is built around matching log line to their source, either a call to a logging function or an exception being
thrown.
Many log lines do not include information about their source to logsmash attempts to find the source of the log by
comparing
the logged message against a list of known log sources included in the binary.
Logsmash is built around matching log line to their source, either a call to a
logging function or an exception being thrown. Many log lines do not include
information about their source to logsmash attempts to find the source of the
log by comparing the logged message against a list of known log sources included
in the binary.
Since multiple log sources can create similar log lines, some lines will match to multiple log sources, in those cases
logsmash
will show all the matched sources as a single item.
Log items that cannot be matched to any known source are grouped together
Since multiple log sources can create similar log lines, some lines will match
to multiple log sources, in those cases logsmash will show all the matched
sources as a single item. Log items that cannot be matched to any known source
are grouped together
## UI overview
@ -50,14 +54,15 @@ Logsmash contains the following UI pages:
### Overview
Shows a list of all matched sources with their source location, the number of log items matched to the source, and a
graph of log occurrence over time.
Shows a list of all matched sources with their source location, the number of
log items matched to the source, and a graph of log occurrence over time.
Selecting an item will bring you to the source overview for the log source.
### Source overview
Shows a list of distinct log lines (lines with the same message, level and context).
Shows a list of distinct log lines (lines with the same message, level and
context).
Selecting an item will bring you to the log list
@ -69,22 +74,23 @@ Selecting an item will bring you to the log page
### Log page
Shows the message and accompanying metadata for the log line. For log lines with exceptions, the exception backtrace is
shown.
Shows the message and accompanying metadata for the log line. For log lines with
exceptions, the exception backtrace is shown.
### Log occurrence graph
Lists that contain grouped log lines come with a graph that show how often the group of log lines occurred over time as
a histogram.
Each line in the list will show a small version of the graph, and a larger version of the graph for the selected item is
shown above the list.
Lists that contain grouped log lines come with a graph that show how often the
group of log lines occurred over time as a histogram. Each line in the list will
show a small version of the graph, and a larger version of the graph for the
selected item is shown above the list.
## Keyboard controls
- All lists are navigated with the arrow keys, `PgUp`/`PgDown` and `Home`/`End`.
- Items in lists can be selected with `Enter` or the right arrow key.
- You can return to the previous page with `Esc` or the left arrow key.
- Single log items can be copied as json with `c` (requires [OSC 52 support](https://github.com/ojroques/vim-oscyank))
- Single log items can be copied as json with `c` (requires
[OSC 52 support](https://github.com/ojroques/vim-oscyank))
## Supported data
@ -119,14 +125,16 @@ Currently, the program can match against data from the following sources:
## Building
The preferred way of building is trough [`nix`](https://nixos.org/download/#download-nix), this ensures that all baked
The preferred way of building is trough
[`nix`](https://nixos.org/download/#download-nix), this ensures that all baked
data is automatically up-to-date.
`nix build .#logsmash`
#### Updating baked data
Note that this is only needed when building with cargo, building with nix automatically uses the latest data.
Note that this is only needed when building with cargo, building with nix
automatically uses the latest data.
```bash
rm -r data/src/data

33
flake.nix
View file

@ -10,21 +10,22 @@
inputs.flakelight.follows = "flakelight";
};
};
outputs = { mill-scale, ... }: mill-scale ./. {
crossTargets = [
"x86_64-unknown-linux-musl"
"x86_64-pc-windows-gnu"
];
packageOpts = { extracted-logs-rust, ... }: {
preBuild = ''
rm -r -f data/src/data
cp -r ${extracted-logs-rust} data/src/data
'';
outputs = {mill-scale, ...}:
mill-scale ./. {
crossTargets = [
"x86_64-unknown-linux-musl"
"x86_64-pc-windows-gnu"
];
packageOpts = {extracted-logs-rust, ...}: {
preBuild = ''
rm -r -f data/src/data
cp -r ${extracted-logs-rust} data/src/data
'';
};
withOverlays = import ./nix/overlay.nix;
packages = {
logsmash = {logsmash}: logsmash;
extracted-logs-rust = {extracted-logs-rust}: extracted-logs-rust;
};
};
withOverlays = import ./nix/overlay.nix;
packages = {
logsmash = { logsmash }: logsmash;
extracted-logs-rust = { extracted-logs-rust }: extracted-logs-rust;
};
};
}

13
nix/data-mod.nix
View file

@ -1,9 +1,8 @@
{ packages
, lib
, writeTextDir
,
}:
let
{
packages,
lib,
writeTextDir,
}: let
inherit (builtins) head attrNames concatStringsSep replaceStrings;
inherit (lib.attrsets) mapAttrsToList;
inherit (lib.lists) flatten last;
@ -35,4 +34,4 @@ let
}
'';
in
writeTextDir "mod.rs" code
writeTextDir "mod.rs" code

60
nix/extracted-logs.nix
View file

@ -1,34 +1,36 @@
{ stdenvNoCC
, logging-extractor
, fetchzip
, name
, version
, url
, major
, sha256
, mode ? "json"
,
}:
let
ext = if mode == "rust" then "rs" else "json";
cleanedMajor = builtins.replaceStrings [ "." ] [ "_" ] major;
{
stdenvNoCC,
logging-extractor,
fetchzip,
name,
version,
url,
major,
sha256,
mode ? "json",
}: let
ext =
if mode == "rust"
then "rs"
else "json";
cleanedMajor = builtins.replaceStrings ["."] ["_"] major;
in
stdenvNoCC.mkDerivation rec {
pname = "extractor-logs-${name}-${cleanedMajor}";
inherit version;
stdenvNoCC.mkDerivation rec {
pname = "extractor-logs-${name}-${cleanedMajor}";
inherit version;
src = fetchzip {
inherit url sha256;
};
src = fetchzip {
inherit url sha256;
};
nativeBuildInputs = [ logging-extractor ];
nativeBuildInputs = [logging-extractor];
buildPhase = ''
logging-extractor . ${mode} > logs.${ext}
'';
buildPhase = ''
logging-extractor . ${mode} > logs.${ext}
'';
installPhase = ''
mkdir -p $out
cp logs.* $out/${name}_${cleanedMajor}.${ext}
'';
}
installPhase = ''
mkdir -p $out
cp logs.* $out/${name}_${cleanedMajor}.${ext}
'';
}

29
nix/logging-extractor.nix
View file

@ -1,19 +1,18 @@
{ stdenv
, rustPlatform
, lib
,
}:
let
{
stdenv,
rustPlatform,
lib,
}: let
inherit (lib.sources) sourceByRegex;
src = sourceByRegex ../logging-extractor [ "Cargo.*" "(src|tests|test-data)(/.*)?" ];
src = sourceByRegex ../logging-extractor ["Cargo.*" "(src|tests|test-data)(/.*)?"];
in
rustPlatform.buildRustPackage rec {
pname = "logging-extractor";
version = "0.1.0";
rustPlatform.buildRustPackage rec {
pname = "logging-extractor";
version = "0.1.0";
inherit src;
inherit src;
cargoLock = {
lockFile = ../logging-extractor/Cargo.lock;
};
}
cargoLock = {
lockFile = ../logging-extractor/Cargo.lock;
};
}

39
nix/logsmash.nix
View file

@ -1,29 +1,28 @@
{ stdenv
, makeRustPlatform
, rust-bin
, lib
, extracted-logs-rust
,
}:
let
{
stdenv,
makeRustPlatform,
rust-bin,
lib,
extracted-logs-rust,
}: let
inherit (lib) sourceByRegex readFile;
src = sourceByRegex ../. [ "Cargo.*" "(src|data)(/.*)?" ];
src = sourceByRegex ../. ["Cargo.*" "(src|data)(/.*)?"];
rustPlatform = makeRustPlatform {
cargo = rust-bin.stable.latest.default;
rustc = rust-bin.stable.latest.default;
};
version = (fromTOML (readFile ../Cargo.toml)).package.version;
in
rustPlatform.buildRustPackage rec {
pname = "logsmash";
inherit version src;
rustPlatform.buildRustPackage rec {
pname = "logsmash";
inherit version src;
preBuild = ''
rm -r data/src/data
cp -r ${extracted-logs-rust} data/src/data
'';
preBuild = ''
rm -r data/src/data
cp -r ${extracted-logs-rust} data/src/data
'';
cargoLock = {
lockFile = ../Cargo.lock;
};
}
cargoLock = {
lockFile = ../Cargo.lock;
};
}

14
nix/overlay.nix
View file

@ -1,5 +1,4 @@
final: prev:
let
final: prev: let
inherit (builtins) mapAttrs attrValues map tail;
inherit (prev) symlinkJoin;
inherit (prev.lib) importJSON;
@ -16,19 +15,18 @@ let
inherit (data) url sha256 version;
inherit name major mode;
}));
in
{
logging-extractor = final.callPackage ./logging-extractor.nix { };
in {
logging-extractor = final.callPackage ./logging-extractor.nix {};
extracted-logs-parts = mapAttrs (loggingFor "json") packages;
extracted-logs-parts-rust = mapAttrs (loggingFor "rust") packages;
extracted-logs-rust-mod = final.callPackage ./data-mod.nix { inherit packages; };
extracted-logs-rust-mod = final.callPackage ./data-mod.nix {inherit packages;};
extracted-logs = symlinkJoin {
name = "extracted-logs";
paths = flatten (map attrValues (attrValues final.extracted-logs-parts));
};
extracted-logs-rust = symlinkJoin {
name = "extracted-logs";
paths = (flatten (map attrValues (attrValues final.extracted-logs-parts-rust))) ++ [ final.extracted-logs-rust-mod ];
paths = (flatten (map attrValues (attrValues final.extracted-logs-parts-rust))) ++ [final.extracted-logs-rust-mod];
};
logsmash = final.callPackage ./logsmash.nix { };
logsmash = final.callPackage ./logsmash.nix {};
}