OSSP L2 ======= RSE: - channel API cleanup: open semantics - bind and udp parameters in socket channel - socket channel convert to use SA library - prefix channels: - printf style without strftime; - l2tool [addr:]port - autoflush via shared memory and sub-process? - IRC channel - NNTP channel - SNMP trap channel - perhaps generalize channels and get rid of stream by putting formatters into external object referenced by channels and by providing a T-channel at the top. - perhaps replace too large PCRE stuff with smaller pattern matching stuff - file channel should optionally support file locking via fcntl(3). - syslog channel should support direct remote logging - implement "filter" channel MS: - implement pipe channel - review pipe handler for dangling descriptors - configure only checks existance of command in non-shell mode - find alternative to exec arguments which is hard coded to 256 - signal handler chaining, save old signal handler and call it after our own ISSUES ------ o hook_write's should perhaps receive a nul-termined string instead of buf+size, because syslog else has to re-buffer it in order to append the nul terminator character. o Stream Members: channels static array - consider dynamic o buffer user needs to know how a buffer object behaves in relation to up/downstream channels. When does it pass its data to the next channel, when does it erase, what happens to its data when it is over written or flushed... o syslog many options need docu, and we should mention to the user that more info is found in the man page for syslog(), because after all that is what is doing all the work in our implementation. Also, can we really properly document these features if they change from one system's syslog to the next? o errors when a channel fails during an operation, how does it report this? How should a user interpret the error message or other data? Do we need more accurate or detailed error messages in the channel code? When a channel fails, does it continue passing data on to downstream channels? Is it corrupt data? o Syslog Kanal - Trim down to what will be used, right now the channel supports ALL functionality through syslog(3) BRAINSTORMING ------------- Braindump: - debugging is special case of logging - tracing is special case of debugging Channel Handler Configuration: o l2_handler_null - no configuration at all o l2_handler_fd - mode="unix|stdio" - fd=int|FILE* o l2_handler_file - mode="unix|stdio" - path=char* - append="yes|no" o l2_handler_pipe - url="prg:/path/to/program" - fd=int o l2_handler_socket - url="tcp://hostname:port" - fd=int o l2_handler_syslog - ident=char* - should have its own logic and not use unix lib syslog() thus able to write to a remote syslog daemon o l2_handler_filter - pattern=char* o l2_handler_prefix - prefix=char* o l2_handler_buffer - size=size_t o all output channels - should they have downstream, or be true endpoints? o l2_ch_socket - write should handle partial send() thus check the return of send o l2_ch_buffer - write() must implicitly flush() when incoming data is larger than remaining buffer capacity License: - ISC/MIT/BSD Sprache: - C++ - C Aufbau: 1. Layer C++ API log.hpp, log.cpp 2. Layer C API log.h log.c 3. Layer C Backend backend.h backend.c - "make striptease" - optimierung: log(..) { } : log(....); : API Levels: - PANIC (-> LOG_EMERG) - CRITICAL (-> LOG_CRIT) - ERROR (-> LOG_ERR) - WARNING - NOTICE - INFO - TRACE (-> LOG_DEBUG) - DEBUG (-> LOG_DEBUG) - ALERT Level Entscheidungen: >= (default) <= = ; * Backend Channels: 1 Level -> N Channels - file (append) - program (stdin) - syslog - stderr/stdout - null (discard, nicht nur /dev/null) - filedescriptor (escape/ext) - callback function Log Messages: - raw - optional prefixes (inclusive order): string facility level timestamp pid (tid) - errno (like syslog %m) - eigene %{foo}x mit callback function mit context - automatisch: number -> string mapping (fuer error strings) - __FILE__, __LINE__, (__FUNCTION__) Configuration: - ueber C/C++ API - zusaetzlich Config-File 1. /etc/liblog.conf 2. (in ., .., ../..) 3. $HOME/.liblog.conf - !debug -> !code API C (ala MM): - reentrant: log_xxx - non-reentrant: Log_xxx Message Filtering/Masking: - facility und/oder levels und/oder wildcard pattern API Using: - C++: LogManager logm; logm.debug1("test"); logm.configure(" - C: log lh; lh = log_init(LOG_CFGFILE|LOG_CFGPARENT|LOG_XXX|..., "foo" (=facility)); log_configure(lh, "foo", LOG_WARN|LOG_LESSER, null); log_cb(lh, "x", func, ctx); int func(void *ctx, char *str, ...); log_msg(lh, LOG_WARN, "..%{foo}x %s...%E..", cp); log_dbg(lh, "..%{foo}x %s...%E..", cp); log_kill(lh); - Buffered I/O: fuer manche channels non-buffered (debug, errors) fuer manche andere aber buffered (access log, performance) loesung: I/O ueber callbacks (3x: open, write, close) z.B. RRDTool - Varargs: log ist nur wrapper fuer vlog - Error Handling: o log kein Return Code o aber error callback function (dadrin in C++: throw, in C: exit) - Newline Handling: option fuer channel: \r, \r\n, \n oder gleich string und moeglichkeit gar nix (string="") - Perhaps: optionally reopen logfile on each write - An optional syslog(3) compatible API for converting syslog-only based applications (like sendmail) to (restricted) liblog-based applications. - Ein Wort noch zu variablen Argumentlisten in cpp-Makros: gcc unterstützt dies in in der GNU- und der C99-Ausführung. Das heißt, der "..." Parameter kann im Makro respektive über "args" und über "__VA_ARGS__" angesprochen werden. Wichtig ist dabei, daß "..." nicht leer sein -- also kein Argument enthalten -- darf, da sonst der Präprocessor an einem eventuell vorhandenen Komma scheitert. Dies kann beim gcc durch Voranstellen von "##" vor dem "__VA_ARGS__" umgangen werden. Ouch. Beide Erweiterungen sind derzeit nicht aktiv, wenn mit -ansi compiliert wird. Explizit anschalten läßt sich die standard-konforme Erweiterung über "-std=c9x", bzw. "-std=c99" bei neueren gccs. High-Level Configuration Interface ================================== Config-File (OSSP): logging { channel 0 null ALL; channel 1 prefix CUSTOM1 { prefix="%Y-%m-%d/%H:%M:%S" }; channel 2 buffer { size=8k }; channel 3 file { path=/path/to/ossp.stat.log; mode=0644 }; channel 4 prefix wmask=DEBUG { prefix="%Y-%m-%d/%H:%M:%S %L" }; channel 5 buffer { size=4k }; channel 6 file { path=/path/to/ossp.error.log; mode=0644 }; channel 7 syslog wmask=ERROR { peer=remote; host=loghost }; channel 8 filter { regex="(memory|foo)" }; channel 9 smtp { host=loghost; rcpt="rse@engelschall.com" }; stream { 0->(4,7); 4->(5,7); 5->6; 8->9; 1->2->3 }; log access 0; log stat 1; }; Alternative: log access { error: syslog; prefix -> { buffer(size=4k) -> file(path=/path/to/log, mode=0644); panic: smtp(host=en1, rcpt=rse@engelschall.com); } } Command-Line (lmtp2nntp): $ lmtp2nntp -l 'INFO:' { prefix(prefix="%Y-%m-%d/%H:%M:%S %L") ->{ buffer(size=4k) ->file(path=/path/to/log,mode=0644); smtp(host=en1,rcpt=rse@engelschall.com) }; error:syslog } Grammar ======= hierarchy : stream | '{' stream_list '}' ; stream_list : hierarchy | hierarchy ';' stream_list ; stream : channel | channel "->" hierarchy ; channel : channel_mask ':' channel_spec | channel_spec ; channel_mask : T_ID_LEVEL | '(' channel_levels ')' ; channel_levels : T_ID_LEVEL | T_ID_LEVEL '|' channel_mask_list ; channel_spec : T_ID_CHANNEL | T_ID_CHANNEL '(' channel_params ')' ; channel_params : channel_param | channel_param ',' channel_params ; channel_param : T_ID_PARAM '=' T_STRING ;