Index: ossp-pkg/sio/BRAINSTORM.fig RCS File: /v/ossp/cvs/ossp-pkg/sio/BRAINSTORM.fig,v co -q -kk -p'1.1' '/v/ossp/cvs/ossp-pkg/sio/BRAINSTORM.fig,v' | diff -u /dev/null - -L'ossp-pkg/sio/BRAINSTORM.fig' 2>/dev/null --- ossp-pkg/sio/BRAINSTORM.fig +++ - 2024-05-15 09:15:08.620844517 +0200 @@ -0,0 +1,241 @@ +#FIG 3.2 +Landscape +Center +Inches +Letter +181.80 +Single +-2 +1200 2 +0 32 #cccccc +0 33 #f0f0f0 +6 2700 1950 3450 2250 +6 2700 1950 3450 2250 +2 2 0 1 0 0 100 0 20 0.000 0 0 -1 0 0 5 + 2700 1950 2775 1950 2775 2250 2700 2250 2700 1950 +2 2 0 1 0 33 101 0 20 0.000 0 0 -1 0 0 5 + 2775 1950 3375 1950 3375 2250 2775 2250 2775 1950 +2 2 0 1 0 0 100 0 20 0.000 0 0 -1 0 0 5 + 3375 1950 3450 1950 3450 2250 3375 2250 3375 1950 +-6 +-6 +6 1350 1875 1800 2325 +2 2 0 1 0 0 100 0 20 0.000 0 0 -1 0 0 5 + 1725 1950 1800 1950 1800 2250 1725 2250 1725 1950 +2 3 0 1 0 33 101 0 20 0.000 0 0 -1 0 0 5 + 1725 1950 1350 1875 1350 2325 1725 2250 1725 1950 +-6 +6 3900 1950 4650 2250 +6 3900 1950 4650 2250 +2 2 0 1 0 0 100 0 20 0.000 0 0 -1 0 0 5 + 3900 1950 3975 1950 3975 2250 3900 2250 3900 1950 +2 2 0 1 0 33 101 0 20 0.000 0 0 -1 0 0 5 + 3975 1950 4575 1950 4575 2250 3975 2250 3975 1950 +2 2 0 1 0 0 100 0 20 0.000 0 0 -1 0 0 5 + 4575 1950 4650 1950 4650 2250 4575 2250 4575 1950 +-6 +-6 +6 1500 3300 1950 3750 +2 2 0 1 0 0 100 0 20 0.000 0 0 -1 0 0 5 + 1875 3375 1950 3375 1950 3675 1875 3675 1875 3375 +2 3 0 1 0 33 101 0 20 0.000 0 0 -1 0 0 5 + 1875 3375 1500 3300 1500 3750 1875 3675 1875 3375 +-6 +6 1500 3900 1950 4350 +2 2 0 1 0 0 100 0 20 0.000 0 0 -1 0 0 5 + 1875 3975 1950 3975 1950 4275 1875 4275 1875 3975 +2 3 0 1 0 33 101 0 20 0.000 0 0 -1 0 0 5 + 1875 3975 1500 3900 1500 4350 1875 4275 1875 3975 +-6 +6 1500 4500 1950 4950 +2 2 0 1 0 0 100 0 20 0.000 0 0 -1 0 0 5 + 1875 4575 1950 4575 1950 4875 1875 4875 1875 4575 +2 3 0 1 0 33 101 0 20 0.000 0 0 -1 0 0 5 + 1875 4575 1500 4500 1500 4950 1875 4875 1875 4575 +-6 +6 2025 3975 2775 4275 +6 2025 3975 2775 4275 +2 2 0 1 0 0 100 0 20 0.000 0 0 -1 0 0 5 + 2025 3975 2100 3975 2100 4275 2025 4275 2025 3975 +2 2 0 1 0 33 101 0 20 0.000 0 0 -1 0 0 5 + 2100 3975 2700 3975 2700 4275 2100 4275 2100 3975 +2 2 0 1 0 0 100 0 20 0.000 0 0 -1 0 0 5 + 2700 3975 2775 3975 2775 4275 2700 4275 2700 3975 +-6 +-6 +6 2025 4500 2475 4950 +2 2 0 1 0 0 100 0 20 0.000 0 0 -1 0 0 5 + 2100 4575 2025 4575 2025 4875 2100 4875 2100 4575 +2 3 0 1 0 33 101 0 20 0.000 0 0 -1 0 0 5 + 2100 4575 2475 4500 2475 4950 2100 4875 2100 4575 +-6 +6 1500 5100 1950 5550 +2 2 0 1 0 0 100 0 20 0.000 0 0 -1 0 0 5 + 1875 5175 1950 5175 1950 5475 1875 5475 1875 5175 +2 3 0 1 0 33 101 0 20 0.000 0 0 -1 0 0 5 + 1875 5175 1500 5100 1500 5550 1875 5475 1875 5175 +-6 +6 2025 5175 2775 5475 +6 2025 5175 2775 5475 +2 2 0 1 0 0 100 0 20 0.000 0 0 -1 0 0 5 + 2025 5175 2100 5175 2100 5475 2025 5475 2025 5175 +2 2 0 1 0 33 101 0 20 0.000 0 0 -1 0 0 5 + 2100 5175 2700 5175 2700 5475 2100 5475 2100 5175 +2 2 0 1 0 0 100 0 20 0.000 0 0 -1 0 0 5 + 2700 5175 2775 5175 2775 5475 2700 5475 2700 5175 +-6 +-6 +6 2850 5100 3300 5550 +2 2 0 1 0 0 100 0 20 0.000 0 0 -1 0 0 5 + 2925 5175 2850 5175 2850 5475 2925 5475 2925 5175 +2 3 0 1 0 33 101 0 20 0.000 0 0 -1 0 0 5 + 2925 5175 3300 5100 3300 5550 2925 5475 2925 5175 +-6 +6 1500 5700 2250 6000 +6 1500 5700 2250 6000 +2 2 0 1 0 0 100 0 20 0.000 0 0 -1 0 0 5 + 1500 5700 1575 5700 1575 6000 1500 6000 1500 5700 +2 2 0 1 0 33 101 0 20 0.000 0 0 -1 0 0 5 + 1575 5700 2175 5700 2175 6000 1575 6000 1575 5700 +2 2 0 1 0 0 100 0 20 0.000 0 0 -1 0 0 5 + 2175 5700 2250 5700 2250 6000 2175 6000 2175 5700 +-6 +-6 +6 788 3225 1387 3787 +3 3 0 1 0 32 101 0 20 0.000 0 0 0 12 + 975 3300 825 3375 825 3525 825 3600 900 3750 1050 3750 + 1125 3675 1350 3675 1275 3450 1275 3300 1050 3225 975 3300 + -1.000 -1.000 -1.000 -1.000 -1.000 -1.000 -1.000 -1.000 + -1.000 -1.000 -1.000 -1.000 +4 0 0 100 0 20 12 0.0000 4 120 195 968 3555 net\001 +-6 +6 2100 3375 2550 3675 +2 4 0 1 0 32 100 0 20 0.000 0 0 7 0 0 5 + 2550 3675 2100 3675 2100 3375 2550 3375 2550 3675 +4 0 0 100 0 20 12 0.0000 4 120 225 2205 3563 app\001 +-6 +6 900 3900 1350 4350 +2 3 0 1 0 32 101 0 20 0.000 0 0 -1 0 0 6 + 900 4050 900 4350 1200 4350 1200 4050 1125 4050 900 4050 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 2 + 1200 4050 1350 3900 +2 3 0 1 0 32 101 0 20 0.000 0 0 -1 0 0 7 + 900 4050 1200 4050 1200 4350 1350 4200 1350 3900 1050 3900 + 900 4050 +4 0 0 100 0 20 12 0.0000 4 135 180 975 4260 file\001 +-6 +6 863 4500 1462 5062 +3 3 0 1 0 32 101 0 20 0.000 0 0 0 12 + 1050 4575 900 4650 900 4800 900 4875 975 5025 1125 5025 + 1200 4950 1425 4950 1350 4725 1350 4575 1125 4500 1050 4575 + -1.000 -1.000 -1.000 -1.000 -1.000 -1.000 -1.000 -1.000 + -1.000 -1.000 -1.000 -1.000 +4 0 0 100 0 20 12 0.0000 4 120 195 1043 4830 net\001 +-6 +6 900 5700 1350 6000 +2 4 0 1 0 32 100 0 20 0.000 0 0 7 0 0 5 + 1350 6000 900 6000 900 5700 1350 5700 1350 6000 +4 0 0 100 0 20 12 0.0000 4 120 225 1005 5888 app\001 +-6 +6 2400 5700 2850 6000 +2 4 0 1 0 32 100 0 20 0.000 0 0 7 0 0 5 + 2850 6000 2400 6000 2400 5700 2850 5700 2850 6000 +4 0 0 100 0 20 12 0.0000 4 120 225 2505 5888 app\001 +-6 +6 900 5100 1350 5550 +2 3 0 1 0 32 101 0 20 0.000 0 0 -1 0 0 6 + 900 5250 900 5550 1200 5550 1200 5250 1125 5250 900 5250 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 2 + 1200 5250 1350 5100 +2 3 0 1 0 32 101 0 20 0.000 0 0 -1 0 0 7 + 900 5250 1200 5250 1200 5550 1350 5400 1350 5100 1050 5100 + 900 5250 +4 0 0 100 0 20 12 0.0000 4 135 180 975 5460 file\001 +-6 +6 2550 4500 3000 4950 +2 3 0 1 0 32 101 0 20 0.000 0 0 -1 0 0 6 + 2550 4650 2550 4950 2850 4950 2850 4650 2775 4650 2550 4650 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 2 + 2850 4650 3000 4500 +2 3 0 1 0 32 101 0 20 0.000 0 0 -1 0 0 7 + 2550 4650 2850 4650 2850 4950 3000 4800 3000 4500 2700 4500 + 2550 4650 +4 0 0 100 0 20 12 0.0000 4 135 180 2625 4860 file\001 +-6 +6 2925 3975 3375 4275 +2 4 0 1 0 32 100 0 20 0.000 0 0 7 0 0 5 + 3375 4275 2925 4275 2925 3975 3375 3975 3375 4275 +4 0 0 100 0 20 12 0.0000 4 120 225 3030 4163 app\001 +-6 +6 3450 5100 3900 5550 +2 3 0 1 0 32 101 0 20 0.000 0 0 -1 0 0 6 + 3450 5250 3450 5550 3750 5550 3750 5250 3675 5250 3450 5250 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 0 0 2 + 3750 5250 3900 5100 +2 3 0 1 0 32 101 0 20 0.000 0 0 -1 0 0 7 + 3450 5250 3750 5250 3750 5550 3900 5400 3900 5100 3600 5100 + 3450 5250 +4 0 0 100 0 20 12 0.0000 4 135 180 3525 5460 file\001 +-6 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 3300 1275 2700 1875 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 3375 1275 3900 1875 +2 1 2 1 32 7 100 0 -1 1.000 0 0 -1 1 1 2 + 1 0 1.00 60.00 120.00 + 1 0 1.00 60.00 120.00 + 1800 2100 2700 2100 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 3225 1275 1800 1875 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 2550 2775 2250 2175 +2 1 0 1 0 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 1.00 60.00 120.00 + 3450 1275 4575 1875 +2 1 0 4 4 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 2.00 105.00 120.00 + 2175 3150 2025 3450 +2 1 0 4 4 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 2.00 105.00 120.00 + 3000 3750 2850 4050 +2 1 0 4 4 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 2.00 105.00 120.00 + 2175 4350 2025 4650 +2 1 0 4 4 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 2.00 105.00 120.00 + 2925 5025 2775 5325 +2 1 0 4 4 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 2.00 105.00 120.00 + 2400 5625 2250 5925 +2 1 0 4 4 7 100 0 -1 0.000 0 0 -1 1 0 2 + 1 1 2.00 105.00 120.00 + 1275 5625 1500 5850 +4 0 0 100 0 18 12 0.0000 4 180 360 2925 2550 pipe\001 +4 0 0 100 0 18 12 0.0000 4 135 1680 1875 3000 internal connection\001 +4 0 0 100 0 18 24 0.0000 4 45 315 3525 2100 ...\001 +4 0 0 100 0 18 12 0.0000 4 180 360 4050 2550 pipe\001 +4 0 0 100 0 18 12 0.0000 4 180 360 1350 2550 plug\001 +4 0 0 100 0 18 12 0.0000 4 135 300 3150 1200 API\001 +4 0 0 100 0 20 10 0.0000 4 105 210 1800 5925 null\001 +4 0 0 100 0 20 10 0.0000 4 105 240 1575 5400 file1\001 +4 0 0 100 0 20 10 0.0000 4 105 195 2325 5400 zlib\001 +4 0 0 100 0 20 10 0.0000 4 105 240 3000 5400 file2\001 +4 0 0 100 0 20 10 0.0000 4 105 360 1500 4800 socket\001 +4 0 0 100 0 20 10 0.0000 4 105 165 2250 4800 file\001 +4 0 0 100 0 20 10 0.0000 4 105 195 2250 4200 zlib\001 +4 0 0 100 0 20 10 0.0000 4 105 165 1575 4200 file\001 +4 0 0 100 0 20 10 0.0000 4 105 360 1425 3525 socket\001 +4 0 0 100 0 18 12 0.0000 4 135 165 600 5400 4.\001 +4 0 0 100 0 18 12 0.0000 4 135 165 600 5925 5.\001 +4 0 0 100 0 18 12 0.0000 4 135 165 600 4800 3.\001 +4 0 0 100 0 18 12 0.0000 4 135 165 600 4200 2.\001 +4 0 0 100 0 18 12 0.0000 4 135 165 600 3600 1.\001 +4 0 0 100 0 20 12 0.0000 4 165 1965 4050 3600 application reads from a socket\001 +4 0 0 100 0 20 12 0.0000 4 165 2640 4050 4200 application writes data compressed to file\001 +4 0 0 100 0 20 12 0.0000 4 165 2445 4050 4800 application transfers from file to socket\001 +4 0 0 100 0 20 12 0.0000 4 165 2310 4050 5400 application filters data from file to file\001 +4 0 0 100 0 20 12 0.0000 4 165 2400 4050 5925 two applications/threads communicate\001 Index: ossp-pkg/sio/BRAINSTORM.txt RCS File: /v/ossp/cvs/ossp-pkg/sio/BRAINSTORM.txt,v co -q -kk -p'1.1' '/v/ossp/cvs/ossp-pkg/sio/BRAINSTORM.txt,v' | diff -u /dev/null - -L'ossp-pkg/sio/BRAINSTORM.txt' 2>/dev/null --- ossp-pkg/sio/BRAINSTORM.txt +++ - 2024-05-15 09:15:08.623524042 +0200 @@ -0,0 +1,304 @@ + +Names: +o SIO = simple/streams/stacked I/O +o FIO = fast I/O +o IOL = IO Lite (achtung) +o LIO = Lite/Layered IO +o AIO = Abstracted I/O + +Connectors/Plugs: +o Memory + SIO_PLUG *sio_plug_mem(uchar *buf, ulong len, uchar *(cb)(ulong)); + - connect to preallocated buffer + callback function for more buffer allocs + - connect internally to auto-growing buffer(s) +o File + constructors: + - from fd + - from FILE + - from pathname +o Socket + - UDP + - TCP +o FILE*-2-SIO +o fd-2-SIO +o file-2-SIO +o TCP/socket-2-SIO +o UDP/socket-2-SIO +o Pipe/FIFO-2-SIO +o Null + Discard Plug + SIO *sio_plug_null(void) + +Filters: +o Transparent; sio_pipe_trans +o Regex-Matching-Filter +o Regex-Subst-Filter +o MD5/SHA1/DES/IDEA +o Zlib/LZO +o SSL + +IO Models (according to Stevens: Unix Network programming I, p. 144) +o blocking I/O +o nonblocking I/O (NO_HANG, NONBLK) +o I/O multiplexing (select and poll) +o signal driven I/O (SIGIO) +o asynchronous I/O (POSIX aio_xxx) + +Data Structures +o Rings internal +o malloc(3) and mm_malloc(3) aware + +Buffering-Modes: +o flush after timeout, close and exit() + explicit via flush() +o no buffering at all +o Chunking for HTTP Support! +o only read is buffered +o only write is buffered +o read & write is buffered + +Buffering: +o filter koennen window/buffer size einstellen +o filter geben an ob sie shrinken/expandieren/gleichlassen oder random sind +o source connector kann eventuell Buffer vom User direkt nutzen +o umgekehrt kann auch user buffer wird von target connector genutzt werden +o application koennte auch bei write angeben, dasz buffer destroyed + werden darf und dasz er eventuell groesser ist als die daten, + die aus ihm geschrieben werden -> nuetzlich fuer expanding filters + bei der Ausgabe. + +Optimierungen: +o mmap muss moeglich sein +o if target connector = socket & source connector = file => try to use sendfile(2) +o if target connector = file & source connector = file => try to use mmap(2) +o wenn I/O ohne filters dann no buffering oder nur wenn chunks zu klein +o wann es geht writev() und readv() nutzen, um Buffers zu koppeln beim I/O +o sockets haben fuer read/write() einen low water mark (siehe setsockopt) +o FreeBSD 4.0's accept_filter(9) mechanism to check for HTTP request + +NOTES: +o Timeouts muessen generell unterstuetzt werden +o Error handling optional mit callbackfunction+void +o Callback functions for exceptional things +o Socket-Connector muss z.B. shutdown() erlauben +o man sollte meta-data (mtime, inode, etc.) von SIO connector rausfinden + koennen und setzen koennen +o Chunking muss moeglich sein (HTTP?) +o Read sollte lesen bis: + - number of bytes read + - found a char of class [a-z] + - found a string "foo" + - found a regex "..." +o man sollte auf jedem SIO seek()en koennen +o man sollte SIOs in FILEs umwandeln koennen??? +o bei Socket Plugs muss shutdown() also eine Seite moeglich sein + +Architecture: +o three types of objects (siehe SIO.fig): + - plugs (Stecker/Anschluesse) for socket, file, mem, etc. + - pipes (Rohre/Verbindungen) + - pipelines (the Kombination von plugs und pipes zu einer Einheit) +o buffered I/O: es gibt eine buffer-pipe, das ist alles + unbuffered I/O ist also eine Pipleline wo kein buffer-pipe dabei ist + +Conversions: +- ASCII2EBCDIC and vice versa conversions + +o SIO Disciplines for Virtual Filesystem Stuff + URLs as pathname open various network things, tarballs, etc. + file, http, ftp, extfs + ?? vfs, libfetch, libcurl ?? + +Performance Gains: +- use sendfile() +- use TCP_CORK +- use ... + +Filter classes (from Apache discussions): + 1) content-generator (files, CGI, database, etc) + 2) content-filter/munger/processor (SSI, PHP, etc) + 3) content-encoding (gzip?) + 4) digest/message processor? (mod_auth_digest) + 5) transport-encoding (HTTP chunking) + 6) socket-encoding (SSL) + +Fuer Zero-Copy: +The data the user program writes must be page sized and start on a page +boundary in order for it to be run through the zero copy send code. + +Ideas: +http://sourceforge.net/projects/paip + +----------------- + +API Functions: +sio_rc_t sio_attach (sio_t **sio, int fd); +sio_rc_t sio_deattach (sio_t *sio, int *fd); +sio_rc_t sio_setbuffer (sio_t *sio, size_t newsize, size_t *oldsize); +sio_rc_t sio_readvec (sio_t *sio, void **vec, size_t *veclen); +sio_rc_t sio_read (sio_t *sio, void *buf, size_t *buflen); +sio_rc_t sio_readline (sio_t *sio, void *buf, size_t *buflen); +sio_rc_t sio_readchar (sio_t *sio, char *c); +sio_rc_t sio_putback (sio_t *sio, void *buf, size_t buflen); +sio_rc_t sio_undo (sio_t *sio); +sio_rc_t sio_writevec (sio_t *sio, void **vec, size_t veclen); +sio_rc_t sio_write (sio_t *sio, void *buf, size_t buflen); +sio_rc_t sio_writestr (sio_t *sio, char *str); +sio_rc_t sio_writeline (sio_t *sio, void *buf, size_t buflen); +sio_rc_t sio_writechar (sio_t *sio, char c); +sio_rc_t sio_print (sio_t *sio, char *fmt, ...); +sio_rc_t sio_printv (sio_t *sio, char *fmt, va_list ap); +sio_rc_t sio_flush (sio_t *sio); +sio_rc_t sio_error (sio_t *sio, char **error); + +sio_rc_t sio_read (sio_t *sio, sio_ioflags_t flags, ...); +sio_rc_t sio_write(sio_t *sio, sio_ioflags_t flags, ...); + + SIO_CHR | SIO_STR | SIO_BUF | SIO_FMT | SIO_STRVEC | SIO_BUFVEC + type of objects: character, nul-terminated string or buffer+size or + format string based + SIO_MULT + multiple objects are passed in call + SIO_NULLEND + whether size of vector is indicated by NULL or given + + SIO_COPY + objects are copied to library + (internal its SIO_GIFT after copy!) + SIO_GIFT + objects are gifted to library + SIO_LOAN + objects are just loaned to library + +sio_read(sio, SIO_BUF, buf, buflen, &readlen); +sio_read(sio, SIO_LINE, buf, buflen, &readlen); + +sio_write(sio, SIO_STR, "foo"); +sio_write(sio, SIO_STR|SIO_MULT, "foo", "bar", NULL); +sio_write(sio, SIO_VEC|SIO_STR, vec, veclen); +sio_write(sio, SIO_VEC|SIO_STR|SIO_NULLEND, vec); +sio_write(sio, SIO_BUF, buf, buflen); +sio_write(sio, SIO_BUF|SIO_MULT, buf, buflen, buf2, buflen2, NULL); +sio_write(sio, SIO_VEC|SIO_BUF, vec, veclen); +sio_write(sio, SIO_VEC|SIO_BUF|SIO_NULLEND, vec); +sio_write(sio, SIO_STR|SIO_MULT, line, "\r\n", NULL); +sio_write(sio, SIO_CHR, c); +sio_write(sio, SIO_BUF, &c, 1); +sio_write(sio, SIO_FMT, "%c%s%S%b%B", c, cp, cpvec, cpvecsize, buf, bufsize, bufvec, bufvecsize); + +API Comfort: + sio_writestr("..") -> sio_write(SIO_STR, ".."); +API Standard: + sio_write(SIO_STR, char *x) -> sio_output(x, strlen(x)); + sio_write(SIO_VEC, char *x) -> for... sio_output(x.ptr, x.len); done +API Basic: + sio_output + +1. Was ist mit seekable fds (files!)? + seek, tell, +2. Top-level filtering and chaining? + tie, untie + +----------------- +IDEA: + - SIO (Socket IO) + - BIO (Buffered/Filtered I/O) + - BA/BB (Buffer Aggregates, Bucket Brigades -- ACT) + +----------------- + +brigate ::= bucket * +bucket ::= + +----------------- + +/* SFIO: sfreserve? + * sfpool ? + */ + +/* + * Data structures + */ + +/* the general SIO API */ +typedef struct sio_st sio_t; + +/* I/O vector entry (similar to POSIX struct iovec) for sio_{read,write}v() */ +typdef struct sio_iovec_st + char *iov_base; /* Base address. */ + size_t iov_len; /* Length. */ +} sio_iovec_t; + +typedef long sio_off_t; + +typedef sio_uint8_t; +typedef sio_uint16_t; +typedef sio_uint32_t; + +#define SIO_SEEK_SET +#define SIO_SEEK_CUR +#define SIO_SEEK_END + +/* + * Values + */ +#define SIO_EOF (-1) + +/* + * Stream Disciplines + */ +sio_disc_t *sio_disc_null (void); +sio_disc_t *sio_disc_anon (void); +sio_disc_t *sio_disc_fd (int fd, ...); +sio_disc_t *sio_disc_socket (int fd, int type /*tcp,udp*/, ...); +sio_disc_t *sio_disc_pipe (int fd, ...); +sio_disc_t *sio_disc_file (FILE *fp, ...); +sio_disc_t *sio_disc_url (const char *url, ...); + +/* + * Stream Handling + */ +sio_t *sio_new (sio_disc_t *disc); +sio_t *sio_dup (sio_t *sio); +int sio_free (sio_t *sio); + +/* + * I/O Operations + */ +sio_size_t sio_read (sio_t *sio, void *buf, size_t bytes); +sio_size_t sio_write (sio_t *sio, void *buf, size_t bytes); +sio_size_t sio_writev (sio_t *sio, const sio_iovec_t *iov, int iovcnt); +sio_size_t sio_writev (sio_t *sio, const sio_iovec_t *iov, int iovcnt); +sio_off_t sio_seek (sio_t *sio, sio_off_t offset, int type); +sio_size_t sio_move (sio_t *siow, sio_t *sior, int n, int rsc); +int sio_getc (sio_t *sio); +int sio_putc (sio_t *sio, int c); +int sio_nputc (sio_t *sio, int c, sio_size_t n); +int sio_ungetc (sio_t *sio, int c); +char *sio_getr (sio_t *sio, int rsc, int type); +sio_size_t *sio_putr (sio_t *sio, const char *str, int rsc); + +/* + * Data Formatting + */ +sio_site_t sio_printf (sio_t *sio, const char *fmt, ...); +sio_site_t sio_vprintf (sio_t *sio, const char *fmt, va_list ap); +sio_site_t sio_scanf (sio_t *sio, const char *fmt, ...); +sio_site_t sio_vscanf (sio_t *sio, const char *fmt, va_list ap); + +/* + * Buffering & Synchronization + */ +int sio_sync (sio_t *sio); +int sio_purge (sio_t *sio); +int sio_poll (sio_poll_t *pl, sio_size_t pn, sio_time_t timeout); +int sio_eof (sio_t *sio); + +/* + * Stream Control + */ +long sio_ctrl (sio_t *sio, int cmd, ...); +int sio_top (sio_t *sio); +int sio_push (sio_t *sio, sio_t *top); +int sio_pop (sio_t *sio); +int sio_swap (sio_t *sio1, sio_t *sio2); + Index: ossp-pkg/sio/BRAINSTORM/apache-buff.txt RCS File: /v/ossp/cvs/ossp-pkg/sio/BRAINSTORM/apache-buff.txt,v co -q -kk -p'1.1' '/v/ossp/cvs/ossp-pkg/sio/BRAINSTORM/apache-buff.txt,v' | diff -u /dev/null - -L'ossp-pkg/sio/BRAINSTORM/apache-buff.txt' 2>/dev/null --- ossp-pkg/sio/BRAINSTORM/apache-buff.txt +++ - 2024-05-15 09:15:08.626203397 +0200 @@ -0,0 +1,236 @@ +/* ==================================================================== + * Copyright (c) 1996-1999 The Apache Group. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * + * 3. All advertising materials mentioning features or use of this + * software must display the following acknowledgment: + * "This product includes software developed by the Apache Group + * for use in the Apache HTTP server project (http://www.apache.org/)." + * + * 4. The names "Apache Server" and "Apache Group" must not be used to + * endorse or promote products derived from this software without + * prior written permission. For written permission, please contact + * apache@apache.org. + * + * 5. Products derived from this software may not be called "Apache" + * nor may "Apache" appear in their names without prior written + * permission of the Apache Group. + * + * 6. Redistributions of any form whatsoever must retain the following + * acknowledgment: + * "This product includes software developed by the Apache Group + * for use in the Apache HTTP server project (http://www.apache.org/)." + * + * THIS SOFTWARE IS PROVIDED BY THE APACHE GROUP ``AS IS'' AND ANY + * EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE APACHE GROUP OR + * ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT + * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; + * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED + * OF THE POSSIBILITY OF SUCH DAMAGE. + * ==================================================================== + * + * This software consists of voluntary contributions made by many + * individuals on behalf of the Apache Group and was originally based + * on public domain software written at the National Center for + * Supercomputing Applications, University of Illinois, Urbana-Champaign. + * For more information on the Apache Group and the Apache HTTP server + * project, please see . + * + */ + +#ifndef APACHE_BUFF_H +#define APACHE_BUFF_H + +#ifdef __cplusplus +extern "C" { +#endif + +#ifdef B_SFIO +#include "sfio.h" +#endif + +#include + +/* Reading is buffered */ +#define B_RD (1) +/* Writing is buffered */ +#define B_WR (2) +#define B_RDWR (3) +/* At end of file, or closed stream; no further input allowed */ +#define B_EOF (4) +/* No further output possible */ +#define B_EOUT (8) +/* A read error has occurred */ +#define B_RDERR (16) +/* A write error has occurred */ +#define B_WRERR (32) +#ifdef B_ERROR /* in SVR4: sometimes defined in /usr/include/sys/buf.h */ +#undef B_ERROR +#endif +#define B_ERROR (48) +/* Use chunked writing */ +#define B_CHUNK (64) +/* bflush() if a read would block */ +#define B_SAFEREAD (128) +/* buffer is a socket */ +#define B_SOCKET (256) +#ifdef CHARSET_EBCDIC +#define B_ASCII2EBCDIC 0x40000000 /* Enable conversion for this buffer */ +#define B_EBCDIC2ASCII 0x80000000 /* Enable conversion for this buffer */ +#endif /*CHARSET_EBCDIC*/ + +typedef struct buff_struct BUFF; + +struct buff_struct { + int flags; /* flags */ + unsigned char *inptr; /* pointer to next location to read */ + int incnt; /* number of bytes left to read from input buffer; + * always 0 if had a read error */ + int outchunk; /* location of chunk header when chunking */ + int outcnt; /* number of byte put in output buffer */ + unsigned char *inbase; + unsigned char *outbase; + int bufsiz; + void (*error) (BUFF *fb, int op, void *data); + void *error_data; + long int bytes_sent; /* number of bytes actually written */ + + ap_pool *pool; + +/* could also put pointers to the basic I/O routines here */ + int fd; /* the file descriptor */ + int fd_in; /* input file descriptor, if different */ +#ifdef WIN32 + HANDLE hFH; /* Windows filehandle */ +#endif + + /* transport handle, for RPC binding handle or some such */ + void *t_handle; + +#ifdef B_SFIO + Sfio_t *sf_in; + Sfio_t *sf_out; +#endif +}; + +#ifdef B_SFIO +typedef struct { + Sfdisc_t disc; + BUFF *buff; +} apache_sfio; + +extern Sfdisc_t *bsfio_new(pool *p, BUFF *b); +#endif + +/* Options to bset/getopt */ +#define BO_BYTECT (1) + +/* Stream creation and modification */ +API_EXPORT(BUFF *) ap_bcreate(pool *p, int flags); +API_EXPORT(void) ap_bpushfd(BUFF *fb, int fd_in, int fd_out); +#ifdef WIN32 +API_EXPORT(void) ap_bpushh(BUFF *fb, HANDLE hFH); +#endif +API_EXPORT(int) ap_bsetopt(BUFF *fb, int optname, const void *optval); +API_EXPORT(int) ap_bgetopt(BUFF *fb, int optname, void *optval); +API_EXPORT(int) ap_bsetflag(BUFF *fb, int flag, int value); +API_EXPORT(int) ap_bclose(BUFF *fb); + +#define ap_bgetflag(fb, flag) ((fb)->flags & (flag)) + +/* Error handling */ +API_EXPORT(void) ap_bonerror(BUFF *fb, void (*error) (BUFF *, int, void *), + void *data); + +/* I/O */ +API_EXPORT(int) ap_bread(BUFF *fb, void *buf, int nbyte); +API_EXPORT(int) ap_bgets(char *s, int n, BUFF *fb); +API_EXPORT(int) ap_blookc(char *buff, BUFF *fb); +API_EXPORT(int) ap_bskiplf(BUFF *fb); +API_EXPORT(int) ap_bwrite(BUFF *fb, const void *buf, int nbyte); +API_EXPORT(int) ap_bflush(BUFF *fb); +API_EXPORT(int) ap_bputs(const char *x, BUFF *fb); +API_EXPORT(int) ap_bvputs(BUFF *fb,...); +API_EXPORT_NONSTD(int) ap_bprintf(BUFF *fb, const char *fmt,...) + __attribute__((format(printf,2,3))); +API_EXPORT(int) ap_vbprintf(BUFF *fb, const char *fmt, va_list vlist); + +/* Internal routines */ +API_EXPORT(int) ap_bflsbuf(int c, BUFF *fb); +API_EXPORT(int) ap_bfilbuf(BUFF *fb); + +#ifndef CHARSET_EBCDIC + +#define ap_bgetc(fb) ( ((fb)->incnt == 0) ? ap_bfilbuf(fb) : \ + ((fb)->incnt--, *((fb)->inptr++)) ) + +#define ap_bputc(c, fb) ((((fb)->flags & (B_EOUT|B_WRERR|B_WR)) != B_WR || \ + (fb)->outcnt == (fb)->bufsiz) ? ap_bflsbuf(c, (fb)) : \ + ((fb)->outbase[(fb)->outcnt++] = (c), 0)) + +#else /*CHARSET_EBCDIC*/ + +#define ap_bgetc(fb) ( ((fb)->incnt == 0) ? ap_bfilbuf(fb) : \ + ((fb)->incnt--, (fb->flags & B_ASCII2EBCDIC)\ + ?os_toebcdic[(unsigned char)*((fb)->inptr++)]:*((fb)->inptr++)) ) + +#define ap_bputc(c, fb) ((((fb)->flags & (B_EOUT|B_WRERR|B_WR)) != B_WR || \ + (fb)->outcnt == (fb)->bufsiz) ? ap_bflsbuf(c, (fb)) : \ + ((fb)->outbase[(fb)->outcnt++] = (fb->flags & B_EBCDIC2ASCII)\ + ?os_toascii[(unsigned char)c]:(c), 0)) + +#endif /*CHARSET_EBCDIC*/ +struct child_info { +#ifdef WIN32 + /* + * These handles are used by ap_call_exec to call + * create process with pipe handles. + */ + HANDLE hPipeInputRead; + HANDLE hPipeOutputWrite; + HANDLE hPipeErrorWrite; +#else + /* + * We need to put a dummy member in here to avoid compilation + * errors under certain Unix compilers, like SGI's and HPUX's, + * which fail to compile a zero-sized struct. Of course + * it would be much nicer if there was actually a use for this + * structure under Unix. Aah the joys of x-platform code. + */ + int dummy; +#endif +}; +API_EXPORT(int) ap_bspawn_child(pool *, int (*)(void *, child_info *), void *, + enum kill_conditions, BUFF **pipe_in, BUFF **pipe_out, + BUFF **pipe_err); + +/* enable non-blocking operations */ +API_EXPORT(int) ap_bnonblock(BUFF *fb, int direction); +/* and get an fd to select() on */ +API_EXPORT(int) ap_bfileno(BUFF *fb, int direction); + +/* bflush() if a read now would block, but don't actually read anything */ +API_EXPORT(void) ap_bhalfduplex(BUFF *fb); + +#ifdef __cplusplus +} +#endif + +#endif /* !APACHE_BUFF_H */ Index: ossp-pkg/sio/BRAINSTORM/apache-stackedio.txt RCS File: /v/ossp/cvs/ossp-pkg/sio/BRAINSTORM/apache-stackedio.txt,v co -q -kk -p'1.1' '/v/ossp/cvs/ossp-pkg/sio/BRAINSTORM/apache-stackedio.txt,v' | diff -u /dev/null - -L'ossp-pkg/sio/BRAINSTORM/apache-stackedio.txt' 2>/dev/null --- ossp-pkg/sio/BRAINSTORM/apache-stackedio.txt +++ - 2024-05-15 09:15:08.628811599 +0200 @@ -0,0 +1,997 @@ +[djg: comments like this are from dean] + +This past summer, Alexei and I wrote a spec for an I/O Filters API... +this proposal addresses one part of that -- 'stacked' I/O with buff.c. + +We have a couple of options for stacked I/O: we can either use existing +code, such as sfio, or we can rewrite buff.c to do it. We've gone over +the first possibility at length, though, and there were problems with each +implemenation which was mentioned (licensing and compatibility, +specifically); so far as I know, those remain issues. + +Btw -- sfio will be supported w/in this model... it just wouldn't be the +basis for the model's implementation. + + -- Ed Korthof | Web Server Engineer -- + -- ed@organic.com | Organic Online, Inc -- + -- (415) 278-5676 | Fax: (415) 284-6891 -- + +--------------------------------------------------------------------------- +Stacked I/O With BUFFs + Sections: + + 1.) Overview + 2.) The API + User-supplied structures + API functions + 3.) Detailed Description + The bfilter structure + The bbottomfilter structure + The BUFF structure + Public functions in buff.c + 4.) Efficiency Considerations + Buffering + Memory copies + Function chaining + writev + 5.) Code in buff.c + Default Functions + Heuristics for writev + Writing + Reading + Flushing data + Closing stacks and filters + Flags and Options + +************************************************************************* + Overview + +The intention of this API is to make Apache's BUFF structure modular +while retaining high efficiency. Basically, it involves rewriting +buff.c to provide 'stacked' I/O -- where the data passed through a +series of 'filters', which may modify it. + +There are two parts to this, the core code for BUFF structures, and the +"filters" used to implement new behavior. "filter" is used to refer to +both the sets of 5 functions, as shown in the bfilter structure in the +next section, and to BUFFs which are created using a specific bfliter. +These will also be occasionally refered to as "user-supplied", though +the Apache core will need to use these as well for basic functions. + +The user-supplied functions should use only the public BUFF API, rather +than any internal details or functions. One thing which may not be +clear is that in the core BUFF functions, the BUFF pointer passed in +refers to the BUFF on which the operation will happen. OTOH, in the +user-supplied code, the BUFF passed in is the next buffer down the +chain, not the current one. + +************************************************************************* + The API + + User-supplied structures + +First, the bfilter structure is used in all filters: + typedef struct { + int (*writev)(BUFF *, void *, struct iovect *, int); + int (*read)(BUFF *, void *, char *, int); + int (*write)(BUFF *, void *, const char *, int); + int (*flush)(BUFF *, void *, const char *, int, bfilter *); + int (*transmitfile)(BUFF *, void *, file_info_ptr *); + void (*close)(BUFF *, void *); + } bfilter; + +bfilters are placed into a BUFF structure along with a +user-supplied void * pointer. + +Second, the following structure is for use with a filter which can +sit at the bottom of the stack: + + typedef struct { + void *(*bgetfileinfo)(BUFF *, void *); + void (*bpushfileinfo)(BUFF *, void *, void *); + } bbottomfilter; + + + BUFF API functions + +The following functions are new BUFF API functions: + +For filters: + +BUFF * bcreatestack(pool *p, int flags, struct bfilter *, + struct bbottomfilter *, void *); +BUFF * bpushfilter (BUFF *, struct bfilter *, void *); +BUFF * bpushbuffer (BUFF *, BUFF *); +BUFF * bpopfilter(BUFF *); +BUFF * bpopbuffer(BUFF *); +void bclosestack(BUFF *); + +For BUFFs in general: + +int btransmitfile(BUFF *, file_info_ptr *); +int bsetstackopts(BUFF *, int, const void *); +int bsetstackflags(BUFF *, int, int); + +Note that a new flag is needed for bsetstackflags: +B_MAXBUFFERING + +The current bcreate should become + +BUFF * bcreatebuffer (pool *p, int flags, struct bfilter *, void *); + +************************************************************************* + Detailed Explanation + + bfilter structure + +The void * pointer used in all these functions, as well as those in the +bbottomfilter structure and the filter API functions, is always the same +pointer w/in an individual BUFF. + +The first function in a bfilter structure is 'writev'; this is only +needed for high efficiency writing, generally at the level of the system +interface. In it's absence, multiple writes will be done w/ 'write'. +Note that defining 'writev' means you must define 'write'. + +The second is 'write'; this is the generic writing function, taking a BUFF +* to which to write, a block of text, and the length of that block of +text. The expected return is the number of characters (out of that block +of text) which were successfully processed (rather than the number of +characters actually written). + +The third is 'read'; this is the generic reading function, taking a BUFF * +from which to read data, and a void * buffer in which to put text, and the +number of characters to put in that buffer. The expected return is the +number of characters placed in the buffer. + +The fourth is 'flush'; this is intended to force the buffer to spit out +any data it may have been saving, as well as to clear any data the +BUFF code was storing. If the third argument is non-null, then it +contains more text to be printed; that text need not be null terminated, +but the fourth argument contains the length of text to be processed. The +expected return value should be the number of characters handled out +from the third argument (0 if there are none), or -1 on error. Finally, +the fifth argument is a pointer to the bfilter struct containing this +function, so that it may use the write or writev functions in it. Note +that general buffering is handled by BUFF's internal code, and module +writers should not store data for performance reasons. + +The fifth is 'transmitfile', which takes as its arguments a buffer to +which to write (if non-null), the void * pointer containing configuration +(or other) information for this filter, and a system-dependent pointer +(the file_info_ptr structure will be defined on a per-system basis) +containing information required to print the 'file' in question. +This is intended to allow zero-copy TCP in Win32. + +The sixth is 'close'; this is what is called when the connection is being +closed. The 'close' should not be passed on to the next filter in the +stack. Most filters will not need to use this, but if database handles +or some other object is created, this is the point at which to remove it. +Note that flush is called automatically before this. + + bbottomfilter Structure + +The first function, bgetfileinfo, is designed to allow Apache to get +information from a BUFF struct regarding the input and output sources. +This is currently used to get the input file number to select on a +socket to see if there's data waiting to be read. The information +returned is platform specific; the void * pointer passed in holds +the void * pointer passed to all user-supplied functions. + +The second function, bpushfileinfo, is used to push file information +onto a buffer, so that the buffer can be fully constructed and ready +to handle data as soon as possible after a client has connected. +The first void * pointer holds platform specific information (in +Unix, it would be a pair of file descriptors); the second holds the +void * pointer passed to all user-supplied functions. + +[djg: I don't think I really agree with the distinction here between +the bottom and the other filters. Take the select() example, it's +valid for any layer to define a fd that can be used for select... +in fact it's the topmost layer that should really get to make this +definition. Or maybe I just have your top and bottom flipped. In +any event I think this should be part of the filter structure and +not separate.] + + The BUFF structure + +A couple of changes are needed for this structure: remove fd and +fd_in; add a bfilter structure; add a pointer to a bbottomfilter; +add three pointers to the next BUFFs: one for the next BUFF in the +stack, one for the next BUFF which implements write, and one +for the next BUFF which implements read. + + + Public functions in buff.c + +BUFF * bpushfilter (BUFF *, struct bfilter *, void *); + +This function adds the filter functions from bfilter, stacking them on +top of the BUFF. It returns the new top BUFF, or NULL on error. + +BUFF * bpushbuffer (BUFF *, BUFF *); + +This function places the second buffer on the top of the stack that +the first one is on. It returns the new top BUFF, or NULL on error. + +BUFF * bpopfilter(BUFF *); +BUFF * bpopbuffer(BUFF *); + +Unattaches the top-most filter from the stack, and returns the new +top-level BUFF, or NULL on error or when there are no BUFFs +remaining. The two are synonymous. + +void bclosestack(BUFF *); + +Closes the I/O stack, removing all the filters in it. + +BUFF * bcreatestack(pool *p, int flags, struct bfilter *, + struct bbottomfilter *, void *); + +This creates an I/O stack. It returns NULL on error. + +BUFF * bcreatebuffer(pool *p, int flags, struct bfilter *, void *); + +This creates a BUFF for later use with bpushbuffer. The BUFF is +not set up to be used as an I/O stack, however. It returns NULL +on error. + +int bsetstackopts(BUFF *, int, const void *); +int bsetstackflags(BUFF *, int, int); + +These functions, respectively, set options on all the BUFFs in a +stack. The new flag, B_MAXBUFFERING is used to disable a feature +described in the next section, whereby only the first and last +BUFFs will buffer data. + +************************************************************************* + Efficiency Considerations + + Buffering + +All input and output is buffered by the standard buffering code. +People writing code to use buff.c should not concern themselves with +buffering for efficiency, and should not buffer except when necessary. + +The write function will typically be called with large blocks of text; +the read function will attempt to place the specified number of bytes +into the buffer. + +Dean noted that there are possible problems w/ multiple buffers; +further, some applications must not be buffered. This can be +partially dealt with by turning off buffering, or by flushing the +data when appropriate. + +However, some potential problems arise anyway. The simplest example +involves shrinking transformations; suppose that you have a set +of filters, A, B, and C, such that A outputs less text than it +recieves, as does B (say A strips comments, and B gzips the result). +Then after a write to A which fills the buffer, A writes to B. +However, A won't write enough to fill B's buffer, so a memory copy +will be needed. This continues till B's buffer fills up, then +B will write to C's buffer -- with the same effect. + +[djg: I don't think this is the issue I was really worried about -- +in the case of shrinking transformations you are already doing +non-trivial amounts of CPU activity with the data, and there's +no copying of data that you can eliminate anyway. I do recognize +that there are non-CPU intensive filters -- such as DMA-capable +hardware crypto cards. I don't think they're hard to support in +a zero-copy manner though.] + +The maximum additional number of bytes which will be copied in this +scenario is on the order of nk, where n is the total number of bytes, +and k is the number of filters doing shrinking transformations. + +There are several possible solutions to this issue. The first +is to turn off buffering in all but the first filter and the +last filter. This reduces the number of unnecessary byte copies +to at most one per byte, however it means that the functions in +the stack will get called more frequently; but it is the default +behavior, overridable by setting the B_MAXBUFFERING with +bsetstackflags. Most filters won't involve a net shrinking +transformation, so even this will rarely be an issue; however, +if the filters do involve a net shrinking transformation, for +the sake of network-efficiency (sending reasonably sized blocks), +it may be more efficient anyway. + +A second solution is more general use of writev for communication +between different buffers. This complicates the programing work, +however. + + + Memory copies + +Each write function is passed a pointer to constant text; if any changes +are being made to the text, it must be copied. However, if no changes +are made to the text (or to some smaller part of it), then it may be +sent to the next filter without any additional copying. This should +provide the minimal necessary memory copies. + +[djg: Unfortunately this makes it hard to support page-flipping and +async i/o because you don't have any reference counts on the data. +But I go into a little detail that already in docs/page_io.] + + Function chaining + +In order to avoid unnecessary function chaining for reads and writes, +when a filter is pushed onto the stack, the buff.c code will determine +which is the next BUFF which contains a read or write function, and +reads and writes, respectively, will go directly to that BUFF. + + writev + +writev is a function for efficient writing to the system; in terms of +this API, however, it also works for dealing with multiple blocks of +text without doing unnecessary byte copies. It is not required. + +Currently, the system level writev is used in two contexts: for +chunking and when a block of text is writen which, combined with +the text already in the buffer, would make the buffer overflow. + +writev would be implemented both by the default bottom level filter +and by the chunking filter for these operations. In addition, writev +may, be used, as noted above, to pass multiple blocks of text w/o +copying them into a single buffer. Note that if the next filter does +not implement writev, however, this will be equivalent to repeated +calls to write, which may or may not be more efficient. Up to +IOV_MAX-2 blocks of text may be passed along in this manner. Unlike +the system writev call, the writev in this API should be called only +once, with a array with iovec's and a count as to the number of +iovecs in it. + +If a bfilter defines writev, writev will be called whether or not +NO_WRITEV is set; hence, it should deal with that case in a reasonable +manner. + +[djg: We can't guarantee atomicity of writev() when we emulate it. +Probably not a problem, just an observation.] + +************************************************************************* + Code in buff.c + + Default Functions + +The default actions are generally those currently performed by Apache, +save that they they'll only attempt to write to a buffer, and they'll +return an error if there are no more buffers. That is, you must implement +read, write, and flush in the bottom-most filter. + +Except for close(), the default code will simply pass the function call +on to the next filter in the stack. Some samples follow. + + Heuristics for writev + +Currently, we call writev for chunking, and when we get a enough so that +the total overflows the buffer. Since chunking is going to become a +filter, the chunking filter will use writev; in addition, bwrite will +trigger bwritev as shown (note that system specific information should +be kept at the filter level): + +in bwrite: + + if (fb->outcnt > 0 && nbyte + fb->outcnt >= fb->bufsiz) { + /* build iovec structs */ + struct iovec vec[2]; + vec[0].iov_base = (void *) fb->outbase; + vec[0].iov_len = fb->outcnt; + fb->outcnt = 0; + vec[1].iov_base = (void *)buff; + vec[1].iov_length = nbyte; + return bwritev (fb, vec, 2); + } else if (nbye >= fb->bufsiz) { + return write_with_errors(fb,buff,nbyte); + } + +Note that the code above takes the place of large_write (as well +as taking code from it). + +So, bwritev would look something like this (copying and pasting freely +from the current source for writev_it_all, which could be replaced): + +----- +int bwritev (BUFF * fb, struct iovec * vec, int nvecs) { + if (!fb) + return -1; /* the bottom level filter implemented neither write nor + * writev. */ + if (fb->bfilter.bwritev) { + return bf->bfilter.writev(fb->next, vec, nvecs); + } else if (fb->bfilter.write) { + /* while it's nice an easy to build the vector and crud, it's painful + * to deal with partial writes (esp. w/ the vector) + */ + int i = 0,rv; + while (i < nvecs) { + do { + rv = fb->bfilter.write(fb, vec[i].iov_base, vec[i].iov_len); + } while (rv == -1 && (errno == EINTR || errno == EAGAIN) + && !(fb->flags & B_EOUT)); + if (rv == -1) { + if (errno != EINTR && errno != EAGAIN) { + doerror (fb, B_WR); + } + return -1; + } + fb->bytes_sent += rv; + /* recalculate vec to deal with partial writes */ + while (rv > 0) { + if (rv < vec[i].iov_len) { + vec[i].iov_base = (char *)vec[i].iov_base + rv; + vec[i].iov_len -= rv; + rv = 0; + if (vec[i].iov_len == 0) { + ++i; + } + } else { + rv -= vec[i].iov_len; + ++i; + } + } + if (fb->flags & B_EOUT) + return -1; + } + /* if we got here, we wrote it all */ + return 0; + } else { + return bwritev(fb->next,vec,nvecs); + } +} +----- +The default filter's writev function will pretty much like +writev_it_all. + + + Writing + +The general case for writing data is significantly simpler with this +model. Because special cases are not dealt with in the BUFF core, +a single internal interface to writing data is possible; I'm going +to assume it's reasonable to standardize on write_with_errors, but +some other function may be more appropriate. + +In the revised bwrite (which I'll ommit for brievity), the following +must be done: + check for error conditions + check to see if any buffering is done; if not, send the data + directly to the write_with_errors function + check to see if we should use writev or write_with_errors + as above + copy the data to the buffer (we know it fits since we didn't + need writev or write_with_errors) + +The other work the current bwrite is doing is + ifdef'ing around NO_WRITEV + numerous decisions regarding whether or not to send chunks + +Generally, buff.c has a number of functions whose entire purpose is +to handle particular special cases wrt chunking, all of which could +be simplified with a chunking filter. + +write_with_errors would not need to change; buff_write would. Here +is a new version of it: + +----- +/* the lowest level writing primitive */ +static ap_inline int buff_write(BUFF *fb, const void *buf, int nbyte) +{ + if (fb->bfilter.write) + return fb->bfilter.write(fb->next_writer,buff,nbyte); + else + return bwrite(fb->next_writer,buff,nbyte); +} +----- + +If the btransmitfile function is called on a buffer which doesn't implement +it, the system will attempt to read data from the file identified +by the file_info_ptr structure and use other methods to write to it. + + Reading + +One of the basic reading functions in Apache 1.3b3 is buff_read; +here is how it would look within this spec: + +----- +/* the lowest level reading primitive */ +static ap_inline int buff_read(BUFF *fb, void *buf, int nbyte) +{ + int rv; + + if (!fb) + return -1; /* the bottom level filter is not set up properly */ + + if (fb->bfilter.read) + return fb->bfilter.read(fb->next_reader,buf,nbyte,fb->bfilter_info); + else + return bread(fb->next_reader,buff,nbyte); +} +----- +The code currently in buff_read would become part of the default +filter. + + + Flushing data + +flush will get passed on down the stack automatically, with recursive +calls to bflush. The user-supplied flush function will be called then, +and also before close is called. The user-supplied flush should not +call flush on the next buffer. + +[djg: Poorly written "expanding" filters can cause some nastiness +here. In order to flush a layer you have to write out your current +buffer, and that may cause the layer below to overflow a buffer and +flush it. If the filter is expanding then it may have to add more to +the buffer before flushing it to the layer below. It's possible that +the layer below will end up having to flush twice. It's a case where +writev-like capabilities are useful.] + + Closing Stacks and Filters + +When a filter is removed from the stack, flush will be called then close +will be called. When the entire stack is being closed, this operation +will be done automatically on each filter within the stack; generally, +filters should not operate on other filters further down the stack, +except to pass data along when flush is called. + + Flags and Options + +Changes to flags and options using the current functions only affect +one buffer. To affect all the buffers on down the chain, use +bsetstackopts or bsetstackflags. + +bgetopt is currently only used to grab a count of the bytes sent; +it will continue to provide that functionality. bgetflags is +used to provide information on whether or not the connection is +still open; it'll continue to provide that functionality as well. + +The core BUFF operations will remain, though some operations which +are done via flags and options will be done by attaching appropriate +filters instead (eg. chunking). + +[djg: I'd like to consider filesystem metadata as well -- we only need +a few bits of metadata to do HTTP: file size and last modified. We +need an etag generation function, it is specific to the filters in +use. You see, I'm envisioning a bottom layer which pulls data out of +a database rather than reading from a file.] + +------- + +This file is there so that I do not have to remind myself +about the reasons for Layered IO, apart from the obvious one. + +0. To get away from a 1 to 1 mapping + + i.e. a single URI can cause multiple backend requests, + in arbitrary configurations, such as in paralel, tunnel/piped, + or in some sort of funnel mode. Such multiple backend + requests, with fully layered IO can be treated exactly + like any URI request; and recursion is born :-) + +1. To do on the fly charset conversion + + Be, theoretically, be able to send out your content using + latin1, latin2 or any other charset; generated from static + _and_ dynamic content in other charsets (typically unicode + encoded as UTF7 or UTF8). Such conversion is prompted by + things like the user-agent string, a cookie, or other hints + about the capabilities of the OS, language preferences and + other (in)capabilities of the final receipient. + +2. To be able to do fancy templates + + Have your application/cgi sending out an XML structure of + field/value pair-ed contents; which is substituted into a + template by the web server; possibly based on information + accessible/known to the webserver which you do not want to + be known to the backend script. Ideally that template would + be just as easy to generate by a backend as well (see 0). + +3. On the fly translation + + And other general text and output mungling, such as translating + an english page in spanish whilst it goes through your Proxy, + or JPEG-ing a GIF generated by mod_perl+gd. + +Dw. +--------- + +From dgaudet@arctic.org Fri Feb 20 00:36:52 1998 +Date: Fri, 20 Feb 1998 00:35:37 -0800 (PST) +From: Dean Gaudet +To: new-httpd@apache.org +Subject: page-based i/o +X-Comment: Visit http://www.arctic.org/~dgaudet/legal for information regarding copyright and disclaimer. +Reply-To: new-httpd@apache.org + +Ed asked me for more details on what I mean when I talk about "paged based +zero copy i/o". + +While writing mod_mmap_static I was thinking about the primitives that the +core requires of the filesystem. What exactly is it that ties us into the +filesystem? and how would we abstract it? The metadata (last modified +time, file length) is actually pretty easy to abstract. It's also easy to +define an "index" function so that MultiViews and such can be implemented. +And with layered I/O we can hide the actual details of how you access +these "virtual" files. + +But therein lies an inefficiency. If we had only bread() for reading +virtual files, then we would enforce at least one copy of the data. +bread() supplies the place that the caller wants to see the data, and so +the bread() code has to copy it. But there's very little reason that +bread() callers have to supply the buffer... bread() itself could supply +the buffer. Call this new interface page_read(). It looks something like +this: + + typedef struct { + const void *data; + size_t data_len; /* amt of data on page which is valid */ + ... other stuff necessary for managing the page pool ... + } a_page_head; + + /* returns NULL if an error or EOF occurs, on EOF errno will be + * set to 0 + */ + a_page_head *page_read(BUFF *fb); + + /* queues entire page for writing, returns 0 on success, -1 on + * error + */ + int page_write(BUFF *fb, a_page_head *); + +It's very important that a_page_head structures point to the data page +rather than be part of the data page. This way we can build a_page_head +structures which refer to parts of mmap()d memory. + +This stuff is a little more tricky to do, but is a big win for performance. +With this integrated into our layered I/O it means that we can have +zero-copy performance while still getting the advantages of layering. + +But note I'm glossing over a bunch of details... like the fact that we +have to decide if a_page_heads are shared data, and hence need reference +counting (i.e. I said "queues for writing" up there, which means some +bit of the a_page_head data has to be kept until its actually written). +Similarly for the page data. + +There are other tricks in this area that we can take advantage of -- +like interprocess communication on architectures that do page flipping. +On these boxes if you write() something that's page-aligned and page-sized +to a pipe or unix socket, and the other end read()s into a page-aligned +page-sized buffer then the kernel can get away without copying any data. +It just marks the two pages as shared copy-on-write, and only when +they're written to will the copy be made. So to make this work, your +writer uses a ring of 2+ page-aligned/sized buffers so that it's not +writing on something the reader is still reading. + +Dean + +---- + +For details on HPUX and avoiding extra data copies, see +. + +(note that if you get the postscript version instead, you have to +manually edit it to remove the front page before any version of +ghostscript that I have used will read it) + +---- + +I've been told by an engineer in Sun's TCP/IP group that zero-copy TCP +in Solaris 2.6 occurs when: + + - you've got the right interface card (OC-12 ATM card I think) + - you use write() + - your write buffer is 16k aligned and a multiple of 16k in size + +We currently get the 16k stuff for free by using mmap(). But sun's +current code isn't smart enough to deal with our initial writev() +of the headers and first part of the response. + +---- + +Systems that have a system call to efficiently send the contents of a +descriptor across the network. This is probably the single best way +to do static content on systems that support it. + +HPUX: (10.30 and on) + + ssize_t sendfile(int s, int fd, off_t offset, size_t nbytes, + const struct iovec *hdtrl, int flags); + + (allows you to add headers and trailers in the form of iovec + structs) Marc has a man page; ask if you want a copy. Not included + due to copyright issues. man page also available from + http://docs.hp.com/ (in particular, + http://docs.hp.com:80/dynaweb/hpux11/hpuxen1a/rvl3en1a/@Generic__BookTextView/59894;td=3 ) + +Windows NT: + + BOOL TransmitFile( SOCKET hSocket, + HANDLE hFile, + DWORD nNumberOfBytesToWrite, + DWORD nNumberOfBytesPerSend, + LPOVERLAPPED lpOverlapped, + LPTRANSMIT_FILE_BUFFERS lpTransmitBuffers, + DWORD dwFlags + ); + + (does it start from the current position in the handle? I would + hope so, or else it is pretty dumb.) + + lpTransmitBuffers allows for headers and trailers. + + Documentation at: + + http://premium.microsoft.com/msdn/library/sdkdoc/wsapiref_3pwy.htm + http://premium.microsoft.com/msdn/library/conf/html/sa8ff.htm + + Even less related to page based IO: just context switching: + AcceptEx does an accept(), and returns the start of the + input data. see: + + http://premium.microsoft.com/msdn/library/sdkdoc/pdnds/sock2/wsapiref_17jm.htm + + What this means is you require one less syscall to do a + typical request, especially if you have a cache of handles + so you don't have to do an open or close. Hmm. Interesting + question: then, if TransmitFile starts from the current + position, you need a mutex around the seek and the + TransmitFile. If not, you are just limited (eg. byte + ranges) in what you can use it for. + + Also note that TransmitFile can specify TF_REUSE_SOCKET, so that + after use the same socket handle can be passed to AcceptEx. + Obviously only good where we don't have a persistent connection + to worry about. + +---- + +Note that all this is shot to bloody hell by HTTP-NG's multiplexing. +If fragment sizes are big enough, it could still be worthwhile to +do copy avoidence. It also causes performance issues because of +its credit system that limits how much you can write in a single +chunk. + +Don't tell me that if HTTP-NG becomes popular we will seen vendors +embedding SMUX (or whatever multiplexing is used) in the kernel to +get around this stuff. There we go, Apache with a loadable kernel +module. + +---- + +Larry McVoy's document for SGI regarding sendfile/TransmitFile: +ftp://ftp.bitmover.com/pub/splice.ps.gz +From dgaudet@arctic.org Sun Jun 20 11:07:58 1999 +Path: engelschall.com!mail2news!apache.org!new-httpd-owner-rse+apache=en.muc.de +From: dgaudet@arctic.org (Dean Gaudet) +Newsgroups: en.lists.apache-new-httpd +Subject: mpm update +Date: 19 Jun 1999 07:17:00 +0200 +Organization: Mail2News at engelschall.com +Lines: 104 +Approved: postmaster@m2ndom +Message-ID: +Reply-To: new-httpd@apache.org +NNTP-Posting-Host: en1.engelschall.com +X-Trace: en1.engelschall.com 929769420 64417 141.1.129.1 (19 Jun 1999 05:17:00 GMT) +X-Complaints-To: postmaster@engelschall.com +NNTP-Posting-Date: 19 Jun 1999 05:17:00 GMT +X-Mail2News-Gateway: mail2news.engelschall.com +Xref: engelschall.com en.lists.apache-new-httpd:31056 + +I imported mpm-3 into the apache-2.0 repository (tag mpm-3 if you want +it). + +Then I threw in a bunch of my recent email ramblings, because I'm getting +tired of repeating them, mostly off-list to folks who ask "why doesn't +apache do XYZ?" I intend to be more proactive in this area, because it +can only help. + +Then I ripped up BUFF and broke lots of stuff and put in a first crack at +layering. Info on that below. + +If you check out the tree, and build it (using Configuration.mpm) you +should be able to serve up the top page of the manual, that's all I've +tested so far ;) + +Dean + +goals? we need an i/o abstraction which has these properties: + +- buffered and non-buffered modes + + The buffered mode should look like FILE *. + + The non-buffered mode should look more like read(2)/write(2). + +- blocking and non-blocking modes + + The blocking mode is the "easy" mode -- it's what most module writers + will see. The non-blocking mode is the "hard" mode, this is where + module writers wanting to squeeze out some speed will have to play. + In order to build async/sync hybrid models we need the + non-blocking i/o abstraction. + +- timed reads and writes (for blocking cases) + + This is part of my jihad against asynchronous notification. + +- i/o filtering or layering + + Yet another Holy Grail of computing. But I digress. These are + hard when you take into consideration non-blocking i/o -- you have + to keep lots of state. I expect our core filters will all support + non-blocking i/o, well at least the ones I need to make sure we kick + ass on benchmarks. A filter can deny a switch to non-blocking mode, + the server will have to recover gracefully (ha). + +- copy-avoidance + + Hey what about zero copy a la IO-Lite? After having experienced it + in a production setting I'm no longer convinced of its benefits. + There is an enormous amount of overhead keeping lists of buffers, + and reference counts, and cleanup functions, and such which requires + a lot of tuning to get right. I think there may be something here, + but it's not a cakewalk. + + What I do know is that the heuristics I put into apache-1.3 to choose + writev() at times are almost as good as what you can get from doing + full zero-copy in the cases we *currently* care about. To put it + another way, let's wait another generation to deal with zero copy. + + But sendfile/transmitfile/etc. those are still interesting. + + So instead of listing "zero copy" as a property, I'll list + "copy-avoidance". + +So far? + +- ap_bungetc added +- ap_blookc changed to return the character, rather than take a char *buff +- in theory, errno is always useful on return from a BUFF routine +- ap_bhalfduplex, B_SAFEREAD will be re-implemented using a layer I think +- chunking gone for now, will return as a layer +- ebcdic gone for now... it should be a layer + +- ap_iol.h defined, first crack at the layers... + + Step back a second to think on it. Much like we have fread(3) + and read(2), I've got a BUFF and an ap_iol abstraction. An ap_iol + could use a BUFF if it requires some form of buffering, but many + won't require buffering... or can do a better job themselves. + + Consider filters such as: + - ebcdic -> ascii + - encryption + - compression + These all share the property that no matter what, they're going to make + an extra copy of the data. In some cases they can do it in place (read) + or into a fixed buffer... in most cases their buffering requirements + are different than what BUFF offers. + + Consider a filter such as chunking. This could actually use the writev + method to get its job done... depends on the chunks being used. This + is where zero-copy would be really nice, but we can get by with a few + heuristics. + + At any rate -- the NSPR folks didn't see any reason to included a + buffered i/o abstraction on top of their layered i/o abstraction... so + I feel like I'm not the only one who's thinking this way. + +- iol_unix.c implemented... should hold us for a bit + + + + +From dgaudet@arctic.org Mon Jun 28 19:06:50 1999 +Path: engelschall.com!mail2news!apache.org!new-httpd-owner-rse+apache=en.muc.de +From: dgaudet@arctic.org (Dean Gaudet) +Newsgroups: en.lists.apache-new-httpd +Subject: Re: async routines +Date: 28 Jun 1999 17:33:24 +0200 +Organization: Mail2News at engelschall.com +Lines: 96 +Approved: postmaster@m2ndom +Message-ID: +Reply-To: new-httpd@apache.org +NNTP-Posting-Host: en1.engelschall.com +X-Trace: en1.engelschall.com 930584004 99816 141.1.129.1 (28 Jun 1999 15:33:24 GMT) +X-Complaints-To: postmaster@engelschall.com +NNTP-Posting-Date: 28 Jun 1999 15:33:24 GMT +X-Mail2News-Gateway: mail2news.engelschall.com +Xref: engelschall.com en.lists.apache-new-httpd:31280 + +[hope you don't mind me cc'ing new-httpd zach, I think others will be +interested.] + +On Mon, 28 Jun 1999, Zach Brown wrote: + +> so dean, I was wading through the mpm code to see if I could munge the +> sigwait stuff into it. +> +> as far as I could tell, the http protocol routines are still blocking. +> what does the future hold in the way for async routines? :) I basically +> need a way to do something like.. + +You're still waiting for me to get the async stuff in there... I've done +part of the work -- the BUFF layer now supports non-blocking sockets. + +However, the HTTP code will always remain blocking. There's no way I'm +going to try to educate the world in how to write async code... and since +our HTTP code has arbitrary call outs to third party modules... It'd +have a drastic effect on everyone to make this change. + +But I honestly don't think this is a problem. Here's my observations: + +All the popular HTTP clients send their requests in one packet (or two +in the case of a POST and netscape). So the HTTP code would almost +never have to block while processing the request. It may block while +processing a POST -- something which someone else can worry about later, +my code won't be any worse than what we already have in apache. So +any effort we put into making the HTTP parsing code async-safe would +be wasted on the 99.9% case. + +Most responses fit in the socket's send buffer, and again don't require +async support. But we currently do the lingering_close() routine which +could easily use async support. Large responses also could use async +support. + +The goal of HTTP parsing is to figure out which response object to +send. In most cases we can reduce that to a bunch of common response +types: + +- copying a file to the socket +- copying a pipe/socket to the socket (IPC, CGIs) +- copying a mem region to the socket (mmap, some dynamic responses) + +So what we do is we modify the response handlers only. We teach them +about how to send async responses. + +There will be a few new primitives which will tell the core "the response +fits one of these categories, please handle it". The core will do the +rest -- and for MPMs which support async handling, the core will return +to the MPM and let the MPM do the work async... the MPM will call a +completion function supplied by the core. (Note that this will simplify +things for lots of folks... for example, it'll let us move range request +handling to a common spot so that more than just default_handler +can support it.) + +I expect this to be a simple message passing protocol (pass by reference). +Well rather, that's how I expect to implement it in ASH -- where I'll +have a single thread per-process doing the select/poll stuff; and the +other threads are in a pool that handles the protocol stuff. For your +stuff you may want to do it another way -- but we'll be using a common +structure that the core knows about... and that structure will look like +a message: + + struct msg { + enum { + MSG_SEND_FILE, + MSG_SEND_PIPE, + MSG_SEND_MEM, + MSG_LINGERING_CLOSE, + MSG_WAIT_FOR_READ, /* for handling keep-alives */ + ... + } type; + BUFF *client; + void (*completion)(struct msg *, int status); + union { + ... extra data here for whichver types need it ...; + } x; + }; + +The nice thing about this is that these operations are protocol +independant... at this level there's no knowledge of HTTP, so the same +MPM core could be used to implement other protocols. + +> so as I was thinking about this stuff, I realized it might be neat to have +> 'classes' of non blocking pending work and have different threads with +> differnt priorities hacking on it. Say we have a very high priority +> thread that accepts connectoins, does initial header parsing, and +> sendfile()ing data out. We could have lower priority threads that are +> spinning doing 'harder' BUFF work like an encryption layer or gziping +> content, whatever. + +You should be able to implement this in your MPM easily I think... because +you'll see the different message types and can distribute them as needed. + +Dean + Index: ossp-pkg/sio/BRAINSTORM/att-sfio.txt RCS File: /v/ossp/cvs/ossp-pkg/sio/BRAINSTORM/att-sfio.txt,v co -q -kk -p'1.1' '/v/ossp/cvs/ossp-pkg/sio/BRAINSTORM/att-sfio.txt,v' | diff -u /dev/null - -L'ossp-pkg/sio/BRAINSTORM/att-sfio.txt' 2>/dev/null --- ossp-pkg/sio/BRAINSTORM/att-sfio.txt +++ - 2024-05-15 09:15:08.631848357 +0200 @@ -0,0 +1,2442 @@ + + + +SFIO(3) SFIO(3) + + +NNAAMMEE + ssffiioo - safe/fast string/file input/output + +SSYYNNOOPPSSIISS + #include + + DDAATTAA TTYYPPEESS + Void_t; + Sfoff_t; + Sflong_t; + Sfulong_t; + Sfdouble_t; + + Sfio_t; + + Sfdisc_t; + ssize_t (*Sfread_f)(Sfio_t*, Void_t*, size_t, Sfdisc_t*); + ssize_t (*Sfwrite_f)(Sfio_t*, const Void_t*, size_t, Sfdisc_t*); + Sfoff_t (*Sfseek_f)(Sfio_t*, Sfoff_t, int, Sfdisc_t*); + int (*Sfexcept_f)(Sfio_t*, int, Void_t*, Sfdisc_t*); + + Sffmt_t; + int (*Sffmtext_f)(Sfio_t*, Void_t*, Sffmt_t*); + int (*Sffmtevent_f)(Sfio_t*, int, Void_t*, Sffmt_t*); + + SFIO_VERSION + + BBIITT FFLLAAGGSS + SF_STRING + SF_READ + SF_WRITE + SF_APPEND + SF_LINE + SF_SHARE + SF_PUBLIC + SF_MALLOC + SF_STATIC + SF_IOCHECK + SF_BUFCONST + SF_WHOLE + + OOPPEENNIINNGG//CCLLOOSSIINNGG SSTTRREEAAMMSS + Sfio_t* sfnew(Sfio_t* f, Void_t* buf, size_t size, int fd, int flags); + Sfio_t* sfopen(Sfio_t* f, const char* string, const char* mode); + Sfio_t* sfpopen(Sfio_t* f, const char* cmd, const char* mode); + Sfio_t* sftmp(size_t size); + int sfclose(Sfio_t* f); + + IINNPPUUTT//OOUUTTPPUUTT OOPPEERRAATTIIOONNSS + int sfgetc(Sfio_t* f); + int sfputc(Sfio_t* f, int c); + int sfnputc(Sfio_t* f, int c, int n); + int sfungetc(Sfio_t* f, int c); + + + + + 05 August 1999 1 + + + + + +SFIO(3) SFIO(3) + + + Sfulong_t sfgetu(Sfio_t* f); + int sfputu(Sfio_t* f, Sfulong_t v); + Sflong_t sfgetl(Sfio_t* f); + int sfputl(Sfio_t* f, Sflong_t v); + Sfdouble_t sfgetd(Sfio_t* f); + int sfputd(Sfio_t* f, Sfdouble_t v); + + char* sfgetr(Sfio_t* f, int rsc, int type); + ssize_t sfputr(Sfio_t* f, const char* s, int rsc); + Sfoff_t sfmove(Sfio_t* fr, Sfio_t* fw, Sfoff_t n, int rsc); + + ssize_t sfread(Sfio_t* f, Void_t* buf, size_t n); + ssize_t sfwrite(Sfio_t* f, const Void_t* buf, size_t n); + Sfoff_t sfseek(Sfio_t* f, Sfoff_t offset, int type); + Void_t* sfreserve(Sfio_t* f, ssize_t n, int lock); + + DDAATTAA FFOORRMMAATTTTIINNGG + int sfscanf(Sfio_t* f, const char* format, ...); + int sfsscanf(const char* s, const char* format, ...); + int sfvsscanf(const char* s, const char* format, va_list args); + int sfvscanf(Sfio_t* f, const char* format, va_list args); + + int sfprintf(Sfio_t* f, const char* format, ...); + char* sfprints(const char* format, ...); + int sfsprintf(char* s, int n, const char* format, ...); + int sfvsprintf(char* s, int n, const char* format, va_list args); + int sfvprintf(Sfio_t* f, const char* format, va_list args); + + Sffmt_t; + int (*Sffmtext_f)(Sfio_t* f, Void_t* v, Sffmt_t* fe); + int (*Sffmtevent_f)(Sfio_t* f, int type, Void_t* v, Sffmt_t* fe); + void va_copy(va_list to, va_list fr); + long sffmtversion(Sffmt_t* fe, type); + + BBUUFFFFEERRIINNGG,, SSYYNNCCHHRROONNIIZZAATTIIOONN + Void_t* sfsetbuf(Sfio_t* f, Void_t* buf, size_t size); + int sfsync(Sfio_t* f); + int sfpoll(Sfio_t** flist, int n, int timeout); + Sfio_t* sfpool(Sfio_t* f, Sfio_t* poolf, int mode); + int sfpurge(Sfio_t* f); + + DDIISSCCIIPPLLIINNEE,, EEVVEENNTT HHAANNDDLLIINNGG + Sfdisc_t* sfdisc(Sfio_t* f, Sfdisc_t* disc); + int sfraise(Sfio_t* f, int type, Void_t* data); + ssize_t sfrd(Sfio_t* f, Void_t* buf, size_t n, Sfdisc_t* disc); + ssize_t sfwr(Sfio_t* f, const Void_t* buf, size_t n, Sfdisc_t* disc); + Sfoff_t sfsk(Sfio_t* f, Sfoff_t offset, int type, Sfdisc_t* disc); + + SSTTRREEAAMM CCOONNTTRROOLL + int sfset(Sfio_t* f, int flags, int i); + int sfsetfd(Sfio_t* f, int fd); + Sfio_t* sfstack(Sfio_t* base, Sfio_t* top); + Sfio_t* sfswap(Sfio_t* f1, Sfio_t* f2); + + + + + 05 August 1999 2 + + + + + +SFIO(3) SFIO(3) + + + SSTTRREEAAMM IINNFFOORRMMAATTIIOONN + Sfoff_t sfsize(Sfio_t* f); + Sfoff_t sftell(Sfio_t* f); + ssize_t sfvalue(Sfio_t* f); + int sffileno(Sfio_t* f); + + int sfstacked(Sfio_t* f); + int sfeof(Sfio_t* f); + int sferror(Sfio_t* f); + int sfclrerr(Sfio_t* f); + int sfclrlock(Sfio_t* f); + + int sfnotify(void (*notify)(Sfio_t* f, int type, int fd)); + + MMIISSCCEELLLLAANNEEOOUUSS FFUUNNCCTTIIOONNSS + ssize_t sfslen(); + int sfulen(Sfulong_t v); + int sfllen(Sflong_t v); + int sfdlen(Sfdouble_t v); + ssize_t sfpkrd(int fd, Void_t* buf, size_t n, + int rsc, long tm, int peek); + + FFUULLLL SSTTRRUUCCTTUURREE SSFFIIOO__TT + #include + #define SFNEW(buf,size,file,flags,disc) + + EEXXAAMMPPLLEE DDIISSCCIIPPLLIINNEESS + int sfdcdio(Sfio_t* f, size_t bufsize); + int sfdcdos(Sfio_t* f); + int sfdcfilter(Sfio_t* f, const char* cmd); + int sfdclzw(Sfio_t* f); + int sfdcseekable(Sfio_t* f); + int sfdcslow(Sfio_t* f); + int sfdcsubstream(Sfio_t* f, Sfio_t* parent, + Sfoff_t offset, Sfoff_t extent); + int sfdctee(Sfio_t* f, Sfio_t* tee); + int sfdcunion(Sfio_t* f, Sfio_t** array, int n); + + SSTTDDIIOO--CCOOMMPPAATTIIBBIILLIITTYY + #include + cc ... -lstdio -lsfio + +DDEESSCCRRIIPPTTIIOONN + Sfio is a library of I/O functions to manage buffered + streams. Each Sfio stream is a _f_i_l_e _s_t_r_e_a_m, representing + a file (see open(2)), or a _s_t_r_i_n_g _s_t_r_e_a_m, representing a + memory segment. Beyond the usual I/O operations on + streams, Sfio provides I/O disciplines for extended data + processing, stream stacks for recursive stream processing, + and stream pools for automatic data synchronization. The + sfprintf()/sfscanf() functions allow applications to + define their own formatting patterns as well as to rede- + fine existing patterns. + + + + + 05 August 1999 3 + + + + + +SFIO(3) SFIO(3) + + + A discipline defines analogues of the system calls + read(2), write(2) and lseek(2). Such system calls or + their discipline replacements are used to process stream + data. Henceforth, ``_s_y_s_t_e_m _c_a_l_l'' will mean a system call + or its discipline replacement. + + A system call is said to cause an exception if its return + value is non-positive. Unless overridden by exception + handlers (see sfdisc()), an interrupted system call (errno + == EINTR on UNIX systems) will be automatically reinvoked + to continue the ongoing operation. + + The buffer of a stream is typically a memory segment allo- + cated via malloc(3) or supplied by the application. File + streams may also use memory mapping (mmap(2)) if that is + more efficient. When memory mapping is used, the underly- + ing file should not be truncated while the stream is + active. Memory mapping can be turned off using sfset- + buf(). + + There are three _s_t_a_n_d_a_r_d _s_t_r_e_a_m_s: sfstdin for input (file + descriptor 0 on UNIX systems), sfstdout for normal output + (file descriptor 1), and sfstderr for error output (file + descriptor 2). + + + + DDAATTAA TTYYPPEESS + VVooiidd__tt** + This defines a type suitable to exchange data of unknown + types between application and Sfio. Void_t is a macro + defined as void for ANSI-C and C++ and char for other com- + pilation environments. + + + SSffooffff__tt + This defines an integral type suitable to address the + largest possible file extent. + + + SSffuulloonngg__tt,, SSfflloonngg__tt,, SSffddoouubbllee__tt + These are respectively the largest unsigned integer, + signed integer, and floating point value types on the + local platform. + + + SSffiioo__tt + This defines the stream type. + + + SSffddiisscc__tt + ssssiizzee__tt ((**SSffrreeaadd__ff))((SSffiioo__tt**,, VVooiidd__tt**,, ssiizzee__tt,, SSffddiisscc__tt**)) + + + + + + 05 August 1999 4 + + + + + +SFIO(3) SFIO(3) + + + ssssiizzee__tt ((**SSffwwrriittee__ff))((SSffiioo__tt**,, ccoonnsstt VVooiidd__tt**,, ssiizzee__tt,, + SSffddiisscc__tt**)) + SSffooffff__tt ((**SSffsseeeekk__ff))((SSffiioo__tt**,, SSffooffff__tt,, iinntt,, SSffddiisscc__tt**)) + iinntt ((**SSffeexxcceepptt__ff))((SSffiioo__tt**,, iinntt,, VVooiidd__tt**,, SSffddiisscc__tt**)) + Sfdisc_t defines a stream discipline structure. Sfread_f, + Sfwrite_f and Sfseek_f are the types of discipline func- + tions to replace the system calls: read(2), write(2) and + lseek(2). Sfexcept_f is the type of an event-handling + function. See sfdisc() for more details. + + + SSffffmmtt__tt + iinntt ((**SSffffmmtteexxtt__ff))((SSffiioo__tt**,, VVooiidd__tt**,, SSffffmmtt__tt**)) + iinntt ((**SSffffmmtteevveenntt__ff))((SSffiioo__tt**,, iinntt,, VVooiidd__tt**,, SSffffmmtt__tt**)) + Sffmt_t defines a formatting environment that can be used + to extend scanning and formatting in the sfprint()/sfs- + canf() functions. Sffmtext_f and Sffmtevent_f define the + types of extension functions definable in Sffmt_t. See + Sffmt_t below for more details. + + + SSFFIIOO__VVEERRSSIIOONN + This is a macro value of type long int that defines the + current version number of Sfio. For example, the + Sfio1998's version number is 19980501L (which also indi- + cates its release date). + + + BBIITT FFLLAAGGSS + A number of bit flags control stream operations. They are + set either at stream initialization or by calling sfset(). + Following are the flags: + + SF_STRING: + The stream is memory-based. + + SF_READ, SF_WRITE, SF_APPEND: + Flags SF_READ and SF_WRITE indicate readability and + writability. Flag SF_APPEND asserts that the + stream is a file opened in append mode (see open(2) + and fcntl(2)) so that data is always output at the + end of file. On systems without direct support for + append mode, Sfio uses lseek(2) or its discipline + replacement to approximate this behavior. + + SF_LINE: + The stream is line-oriented. For a SF_WRITE + stream, this means that buffered data is flushed + whenever a new-line character, \n, is output. For + a SF_READ stream, SF_LINE is only significant dur- + ing calls to functions in the sfscanf() family. + SF_LINE is set on initialization of any stream rep- + resenting a terminal device. + + + + + 05 August 1999 5 + + + + + +SFIO(3) SFIO(3) + + + SF_SHARE, SF_PUBLIC: + Flag SF_SHARE means that the underlying file + descriptor is shared by independent entities (for + example, multiple processes). + + For a seekable file stream, SF_SHARE means that the + logical stream and the physical file positions will + be made the same before a system call to perform + physical I/O. There are different possibilities. + If SF_PUBLIC is not set, the physical file position + is made equal to the logical stream position. If + SF_PUBLIC is set, there are two cases. If the + physical file position has changed from its last + known position, the logical stream position is made + equal to the new physical file position. Finally, + if the physical file location remains the same as + its last known position, the physical file position + is made the same as the logical stream position. + + For an unseekable stream (e.g., pipes or terminal + devices), if possible, SF_SHARE means that the + block and record I/O operations (sfread(), + sfwrite(), sfmove(), sfgetr(), sfputr(), sfre- + serve(), sfscanf() and sfvprintf()) will ensure: + (1) after each writing operation, the stream is + synchronized and (2) each reading operation only + reads the requested amount. Note, however, that + (2) is not always possible without proper OS facil- + ities such as recv(2) or streamio(4). + + A standard stream that is seekable will be initial- + ized with SF_SHARE|SF_PUBLIC. + + SF_MALLOC: + The stream buffer was obtained via malloc(3) and + can be reallocated or freed. + + SF_STATIC: + The stream structure should not be freed when + closed (sfclose()). This flag is used by an appli- + cations that allocate their own stream structures. + Such applications must use the header file sfio_t.h + instead of sfio.h. + + SF_IOCHECK: + If the stream has a discipline exception handler, + exceptions will be raised in sfsync(), sfpurge() or + before a system call read(2) or write(2) (see + sfdisc()). + + SF_BUFCONST: + The application guarantees that a stream buffer + obtained via sfreserve() or sfgetr() will not be + modified. This allows Sfio to tune buffer + + + + 05 August 1999 6 + + + + + +SFIO(3) SFIO(3) + + + management and memory maps. For example, a memory- + mapped stream will map with MAP_SHARED on and + PROT_WRITE off so that the file itself will likely + be the backing store for mapped pages. + + SF_WHOLE: + This flag guarantees that data written in any sin- + gle sfwrite() or sfputr() call will always be out- + put as a whole to the output device. This is use- + ful in certain applications (e.g., networking) + where a complex object must be output without being + split in different system calls. Note that the + respective stream still buffers data as the buffer + can accomodate. + + + + OOPPEENNIINNGG//CCLLOOSSIINNGG SSTTRREEAAMMSS + SSffiioo__tt** ssffnneeww((SSffiioo__tt** ff,, VVooiidd__tt** bbuuff,, ssiizzee__tt ssiizzee,, iinntt ffdd,, + iinntt ffllaaggss)) + This function creates or renews a stream. It returns the + new stream on success and NULL on error. + + f: If f is NULL, a new stream is created. Otherwise, + f is reused. In this case, if flags does not have + SF_EOF, f shall be closed via sfclose() before + being reused. During a stream renewal, buffer, + pool and discipline stack are preserved. Note + that, except for SF_STATIC streams, renewing a + stream already closed will result in undefined + behavior. + + buf, size: + These determine a buffering scheme. See sfsetbuf() + for more details. + + fd: If SF_STRING is specified in flags, this is + ignored. Otherwise, fd is a file descriptor (e.g., + from open(2)) to use for raw data I/O. Note that + Sfio supports unseekable file descriptors opened + for both read and write, e.g., sockets. + + flags: This is composed from SF_EOF and bit values defined + in the BBIITT FFLLAAGGSS section. + + + SSffiioo__tt** ssffooppeenn((SSffiioo__tt** ff,, ccoonnsstt cchhaarr** ssttrriinngg,, ccoonnsstt cchhaarr** + mmooddee)) + If string is NULL and f is a file stream that has not per- + formed any I/O operation, sfopen() will change the modes + of f according to mode. In this case, sfopen() returns f + on success and NULL on error. This somewhat unusual usage + of sfopen() is good for changing the predefined modes of + standard streams. + + + + 05 August 1999 7 + + + + + +SFIO(3) SFIO(3) + + + sfopen() is normally used to create a new stream or renew + a stream. In this case, it returns the new stream on suc- + cess and NULL on error. Below are the meanings of the + arguments: + + f: This is treated as in sfnew(). + + string: + This is a file name or a string to perform I/O on. + + mode: This is composed from the set of letters {s, r, w, + +, a, x, b, t}. + + s specifies opening a string stream. string can be + a null-terminated string or NULL. Specifying s + alone is equivalent to specifying sr. If s is not + specified, string defines a file name. + + r and w specify read and write modes. Write mode + creates and/or truncates the given file to make an + empty file. The + modifier indicates that the + stream is opened for both read and write. + + a specifies append mode, i.e., data is always out- + put at end of file. + + x specifies exclusive mode, i.e., a file opened for + writing should not already exist. + + b and t specify binary and text modes. + + + SSffiioo__tt** ssffppooppeenn((SSffiioo__tt** ff,, ccoonnsstt cchhaarr** ccmmdd,, ccoonnsstt cchhaarr** + mmooddee)) + This function opens a stream that corresponds to the + coprocess cmd. The argument mode should be composed from + r, w, and +. The argument f, if not NULL, is a stream to + be renewed (see sfnew()). sfpopen() returns the new + stream or NULL on error. + + The standard input/output of cmd is connected to the + application via a pipe if the stream is opened for writ- + ing/reading. If the stream is opened for both reading and + writing, there will be two different associated file + descriptors, one for each type of I/O (note the effect on + sffileno()). + + On opening a coprocess for writing (i.e., mode contains w + or +), the signal handler for SIGPIPE in the parent appli- + cation will be set to SIG_IGN if it is SIG_DFL at that + time. This protects the parent application from being + accidentally killed on any attempt to write to a coprocess + that closes its reading end. Applications that need to + detect such write errors should use disciplines and + + + + 05 August 1999 8 + + + + + +SFIO(3) SFIO(3) + + + exception handlers (see sfdisc()). + + The command cmd is executed by an _i_n_t_e_r_p_r_e_t_e_r which is + either /bin/sh or an executable command defined by the + environment variable SHELL. In either case, the inter- + preter is invoked with 2 arguments, respectively -c and + the given command cmd. When the interpreter is /bin/sh or + /bin/ksh, sfpopen() may execute the command cmd itself if + there are no shell meta-characters in cmd. + + + SSffiioo__tt** ssffttmmpp((ssiizzee__tt ssiizzee)) + This function creates a stream for temporary data. It + returns the new stream or NULL on error. + + A stream created by sftmp() can be completely or partially + memory-resident. If size is SF_UNBOUND, the stream is a + pure string stream. If size is zero, the stream is a pure + file stream. Otherwise, the stream is first created as a + string stream but when its buffer grows larger than size + or on any attempt to change disciplines, a temporary file + is created. Two environment variables, TMPPATH and + TMPDIR, direct where temporary files are created. TMP- + PATH, if defined, specifies a colon-separated set of + directories to be used in a round-robin fashion to create + files. If TMPPATH is undefined, TMPDIR can be used to + specify a single directory to create files. If neither of + TMPPATH and TMPDIR are defined, /tmp is used. + + + iinntt ssffcclloossee((SSffiioo__tt** ff)) + This function closes the stream f and frees its resources. + SF_STATIC should be used if the stream space is to be pre- + served. If f is the base of a stream stack (see sfs- + tack()), all streams on the stack are closed. If f is a + sfpopen-stream, sfclose() waits until the associated com- + mand terminates and returns its exit status. sfclose() + returns -1 for failure and 0 for success. + + SF_READ|SF_SHARE and SF_WRITE streams are synchronized + before closing (see sfsync()). If f has disciplines, + their exception handlers will be called twice. The first + exception handler call has the type argument as one of + SF_CLOSE or SF_NEW (see sfdisc().) The latter, SF_NEW is + used when a stream is being closed via sfnew() so that it + can be renewed. The second call uses type as SF_FINAL and + is done after all closing operations have succeeded but + before the stream itself is deallocated. In either case, + if the exception handler returns a negative value, + sfclose() will immediately return this value. If the + exception handler returns a positive value, sfclose() will + immediately return a zero value. + + + + + + 05 August 1999 9 + + + + + +SFIO(3) SFIO(3) + + + IINNPPUUTT//OOUUPPUUTT OOPPEERRAATTIIOONNSS + iinntt ssffggeettcc((SSffiioo__tt** ff)) + iinntt ssffppuuttcc((SSffiioo__tt** ff,, iinntt cc)) + These functions read/write a byte from/to stream f. + sfgetc() returns the byte read or -1 on error. sfputc() + returns c on success and -1 on error. + + + ssssiizzee__tt ssffnnppuuttcc((SSffiioo__tt** ff,, iinntt cc,, ssiizzee__tt nn)) + This function attempts to write the byte c to f n times. + It returns the number of bytes actually written or -1 on + failure. + + + iinntt ssffuunnggeettcc((SSffiioo__tt** ff,, iinntt cc)) + This function pushes the byte c back into f. If c matches + the byte immediately before the current position in + buffered data, the current position is simply backed up + (note the effect on sftell() and sfseek()). There is no + theoretical limit on the number of bytes that can be + pushed back into a stream. Pushed back bytes not part of + buffered data will be discarded on any operation that + implies buffer synchronization. sfungetc() returns c on + success and -1 on failure. + + + SSffuulloonngg__tt ssffggeettuu((SSffiioo__tt** ff)) + iinntt ssffppuuttuu((SSffiioo__tt** ff,, SSffuulloonngg__tt vv)) + These functions read and write Sfulong_t values in a com- + pact variable-length portable format. Portability across + a write architecture and a read architecture requires that + the bit order in a byte is the same on both architectures + and the written value is storable in an Sfulong_t on the + read architecture. sfgetu() returns the value read or -1 + on error. sfputu() returns the number of bytes written or + -1 on error. See also sfulen(). + + + SSfflloonngg__tt ssffggeettll((SSffiioo__tt** ff)) + iinntt ssffppuuttll((SSffiioo__tt** ff,, SSfflloonngg__tt vv)) + These functions are similar to sfgetu() and sfputu() but + for reading and writing (signed) Sflong_t values. See + also sfllen(). + + + SSffddoouubbllee__tt ssffggeettdd((SSffiioo__tt** ff)) + iinntt ssffppuuttdd((SSffiioo__tt** ff,, SSffddoouubbllee__tt vv)) + These functions read and write Sfdouble_t values. In this + case, portability depends on the input and output archi- + tectures having the same floating point value representa- + tion. Values are coded and decoded using ldexp(3) and + frexp(3) so they are constrained to the sizes supported by + these functions. See also sfdlen(). + + + + + 05 August 1999 10 + + + + + +SFIO(3) SFIO(3) + + + cchhaarr** ssffggeettrr((SSffiioo__tt** ff,, iinntt rrsscc,, iinntt ttyyppee)) + This function reads a record of data ending in the record + separator rsc. After sfgetr() returns, the length of the + record even if it is incomplete can be retrieved with + sfvalue(). sfgetr() returns the record on success and + NULL on error. + + The type argument is composed of some subset of the below + bit flags: + + SF_STRING: + A null byte will replace the record separator to + make the record into a C string. Otherwise, the + record separator is left alone. + + SF_LOCKR: + Upon successfully obtaining a record r, the stream + will be locked from further access until it is + released with a call sfread(f,r,0). + + SF_LASTR: + This should be used only after a failed sfgetr() to + retrieve the last incomplete record. In this case, + rsc is ignored. + + + ssssiizzee__tt ssffppuuttrr((SSffiioo__tt** ff,, ccoonnsstt cchhaarr** ss,, iinntt rrsscc)) + This function writes the null-terminated string s to f. + If rsc is non-negative, (unsigned char)rsc is output after + the string. sfputr() returns the number of bytes written + or -1 on failure. + + + SSffooffff__tt ssffmmoovvee((SSffiioo__tt** ffrr,, SSffiioo__tt** ffww,, SSffooffff__tt nn,, iinntt rrsscc)) + This function moves objects from input stream fr to output + stream fw. sfmove() returns the number of objects moved + or -1 on failure. + + An object is either a byte or a record. The latter is + indicated by a non-negative value for the record separator + character rsc. If n is negative, all of fr will be moved. + Otherwise, n indicates the number of objects to move. If + either fr or fw is NULL, it acts as if it is a stream cor- + responding to /dev/null, the UNIX device that has no read + data and throws away any write data. For example, the + call sfmove(f,(Sfio_t*)0,(Sfoff_t)(-1),'\n') simply counts + the number of lines in stream f. + + + ssssiizzee__tt ssffrreeaadd((SSffiioo__tt** ff,, VVooiidd__tt** bbuuff,, ssiizzee__tt nn)) + This function reads up to n bytes from f into buffer buf. + It returns the number of bytes actually read or -1 on + error. + + + + + 05 August 1999 11 + + + + + +SFIO(3) SFIO(3) + + + ssssiizzee__tt ssffwwrriittee((SSffiioo__tt** ff,, ccoonnsstt VVooiidd__tt** bbuuff,, ssiizzee__tt nn)) + This function writes n bytes from buf to f. If f is + SF_STRING, and the buffer is not large enough, an SF_WRITE + exception shall be raised. sfwrite() returns the number + of bytes written or -1 on failure. + + + SSffooffff__tt ssffsseeeekk((SSffiioo__tt** ff,, SSffooffff__tt ooffffsseett,, iinntt ttyyppee)) + This function sets a new I/O position for f. It returns + the new position or -1 on failure. + + If the stream is a SF_STRING stream and the new address is + beyond the current buffer extent, an SF_SEEK exception + will be raised (see sfdisc()). + + The new position is determined based on offset and type + which is composed from the bit flags: + + 0 or SEEK_SET: + offset is the desired position. + + 1 or SEEK_CUR: + offset is relative to the current position (see + SF_PUBLIC below). + + 2 or SEEK_END: + offset is relative to the physical end of file. + + SF_SHARE: + The stream is treated as if it has the control bit + SF_SHARE on. This implies that a system call seek + will be done to ensure that the location seeking to + is valid. + + SF_PUBLIC: + The stream is treated as if it has the control bit + SF_PUBLIC on. If the physical file position has + changed from its last known location, the current + position is taken as the new physical position. + Otherwise, the current position is the logical + stream position. + + + VVooiidd__tt** ssffrreesseerrvvee((SSffiioo__tt** ff,, ssssiizzee__tt nn,, iinntt lloocckk)) + This function reserves a data block from f. For a SF_READ + stream, a data block is a segment of data and for a + SF_WRITE stream, it is a buffer suitable for writing. For + consistency, a stream opened with SF_READ|SF_WRITE will be + treated as if it is a SF_READ stream (see sfset() for set- + ting a particular mode.) sfreserve() returns the obtained + data block on success and NULL on failure. + + After a sfreserve() call, whether or not it succeeds, + sfvalue() can be used to obtain the size of the (may-have- + + + + 05 August 1999 12 + + + + + +SFIO(3) SFIO(3) + + + been) available data block. + + n != 0: + f will be filled or flushed as necessary to make + available a data block of size at least the abso- + lute value of n. If this is successful and lock is + non-positive, the I/O position will advance by the + size of the available data block when n is negative + or by exactly n when n is positive. For example, + sfreserve(f,-1,0) returns a positive size data + block and advances the I/O position by its size. + On the other hand, sfreserve(f,1,0) returns a posi- + tive size data block and advances the I/O position + by exactly 1. + + n == 0: + If lock is zero, f will be filled or flushed as + necessary to ensure that a positive-size data block + is available. If lock is non-zero, no fill or + flush will be performed. In addition, if lock is + positive, f will be locked from further access. + Therefore, an application can lock f with sfre- + serve(f,0,1). + + lock: When lock is positive, there are restrictions. If + f is SF_READ and not using memory-mapping (see + sfsetbuf()), reservation must be limited to stream + buffer size. If f is SF_READ|SF_SHARE and unseek- + able, sfreserve() will peek at incoming data using + either recv(2) or streamio(4) without reading + ahead. In this case, if peeking is not supported + by the underlying platform, sfreserve() will fail. + Note that SF_SHARE is automatically on for sfstdin + so programs (e.g., //bbiinn//ssoorrtt) that will consume all + of input anyway should turn this bit off to reduce + the number of system calls. + + If a reservation successfully results in a data + block data, and lock is positive, the stream I/O + position does not advance and f will be locked + until unlocked with sfread/sfwrite(f,data,size). + sfread() should be used on read-only stream and + sfwrite() should be used on write-only stream. A + stream in both read and write modes can release the + lock with either call (with associated operational + semantics!) + + + + DDAATTAA FFOORRMMAATTTTIINNGG + Data printing and scanning are done via the sfprintf() and + sfscanf() family of functions. These functions are simi- + lar to their ANSI-C fprintf() and fscanf() counterparts. + However, the Sfio versions have been extended for both + + + + 05 August 1999 13 + + + + + +SFIO(3) SFIO(3) + + + portability and generality. In particular, a notion of a + formatting environment stack is introduced. Each format- + ting element on the stack defines a separate _f_o_r_m_a_t_t_i_n_g + _p_a_i_r of a format specification string, char* format (the + usual second argument in the formatting functions), and an + argument list, va_list args (the third argument in func- + tions sfvprintf() and sfvscanf()). A formatting environ- + ment element may also specify extension functions to + obtain or assign arguments and to provide new semantics + for pattern processing. To simplify the description + below, whenever we talk about an argument list, unless + noted otherwise, it is understood that this means either + the true argument list when there is no extension function + or the action to be taken by such a function in processing + arguments. The manipulation of the formatting environment + stack is done via the pattern ! discussed below. + + + %%!! aanndd SSffffmmtt__tt + The pattern %! manipulates the formatting environment + stack to (1) change the top environment to a new environ- + ment, (2) stack a new environment on top of the current + top, or (3) pop the top environment. The bottom of the + environment stack always contains a virtual environment + with the original formatting pair and without any exten- + sion functions. + + The top environment of a stack, say fe, is automatically + popped whenever its format string is completely processed. + In this case, its event-handling function (if any) is + called as (*eventf)(f,SF_FINAL,NIL(Void_t*),fe). The top + environment can also be popped by giving an argument NULL + to %! or by returning a negative value in an extension + function. In these cases, the event-handling function is + called as (*eventf)(f,SF_DPOP,form,fe) where form is the + remainder of the format string. A negative return value + from the event handling function will prevent the environ- + ment from being popped. + + A formatting environment is a structure of type Sffmt_t + which contains the following elements: + + Sffmtext_f extf; /* extension processor */ + Sffmtevent_f eventf; /* event handler */ + + char* form; /* format string to stack */ + va_list args; /* corresponding arg list */ + + int fmt; /* pattern being processed */ + ssize_t size; /* object size */ + int flags; /* formatting control flags */ + int width; /* width of field */ + int precis; /* precision required */ + int base; /* conversion base */ + + + + 05 August 1999 14 + + + + + +SFIO(3) SFIO(3) + + + char* t_str; /* type string in parentheses */ + int n_str; /* length of t_str */ + + The first four elements of Sffmt_t must be defined by the + application. The two function fields should not be + changed during processing. Other elements of Sffmt_t are + set on calls to the extension function Sffmt_t.extf and, + in turn, can be modified by this function to redirect for- + matting or scanning. For example, consider a call from a + sfprintf() function to process an unknown pattern %t + (which we may take to mean ``time'') based on a formatting + environment fe. fe->extf may reset fe->fmt to `d' upon + returing to cause sfprintf() to process the value being + formatted as an integer. + + Below are the fields of Sffmt_t: + + extf: extf is a function to extend scanning and format- + ting patterns. Its usage is discussed below. + + eventf: + This is a function to process events as discussed + earlier. + + form and args: + This is the formatting pair of a specification + string and corresponding argument list. When an + environment fe is being inserted into the stack, if + fe->form is NULL, the top environment is changed to + fe and its associated extension functions but pro- + cessing of the current formatting pair continues. + On the other hand, if fe->form is not NULL, the new + environment is pushed onto the stack so that pat- + tern processing will start with the new formatting + pair as well as any associated extension functions. + During processing, whenever extf is called, form + and args will be set to the current values of the + formatting pair in use. + + fmt: This is the pattern being processed. + + size: This is the size of the object being processed. + + flags: This is a collection of bits defining the format- + ting flags specified for the pattern. The bits + are: + + SFFMT_LEFT: Flag - in sfprintf(). + + SFFMT_SIGN: Flag + in sfprintf(). + + SFFMT_BLANK: Flag _s_p_a_c_e in sfprintf(). + + SFFMT_ZERO: Flag 0 in sfprintf(). + + + + 05 August 1999 15 + + + + + +SFIO(3) SFIO(3) + + + SFFMT_THOUSAND: Flag ' in sfprintf(). + + SFFMT_LONG: Flag l in sfprintf() and sfscanf(). + + SFFMT_LLONG: Flag ll in sfprintf() and sfscanf(). + + SFFMT_SHORT: Flag h in sfprintf() and sfscanf(). + + SFFMT_LDOUBLE: Flag L in sfprintf() and sfscanf(). + + SFFMT_IFLAG: flag I in sfprintf() and sfscanf(). + + SFFMT_ALTER: Flag # in sfprintf() and sfscanf(). + + SFFMT_SKIP: Flag * in sfscanf(). + + SFFMT_ARGPOS: This indicates argument processing + for pos$. + + SFFMT_VALUE: This is set by fe->extf to indicate + that it is returning a value to be formatted or the + address of an object to be assigned. + + + width: This is the field width. + + precis: + This is the precision. + + base: This is the conversion base. + + t_str and n_str: + This is the type string and its size. + + + iinntt ((**SSffffmmtteexxtt__ff))((SSffiioo__tt** ff,, VVooiidd__tt** vv,, SSffffmmtt__tt** ffee)) + This is the type of the extension function fe->extf to + process patterns and arguments. Arguments are always pro- + cessed in order and fe->extf is called exactly once per + argument. Note that, when pos$ (below) is not used any- + where in a format string, each argument is used exactly + once per a corresponding pattern. In that case, fe->extf + is called as soon as the pattern is recognized and before + any scanning or formatting. On the other hand, when pos$ + is used in a format string, an argument may be used multi- + ple times. In this case, all arguments shall be processed + in order by calling fe->extf exactly once per argument + before any pattern processing. This case is signified by + the flag SFFMT_ARGPOS in fe->flags. + + In addition to the predefined formatting patterns and + other application-defined patterns, fe->extf may be called + with fe->fmt being one of `(' (left parenthesis), `.' + (dot), and `I'. The left parenthesis requests a string to + + + + 05 August 1999 16 + + + + + +SFIO(3) SFIO(3) + + + be used as the type string discussed below. In this case, + upon returning, fe->extf should set the fe->size field to + be the length of the string or a negative value to indi- + cate a null-terminated string. The dot requests an inte- + ger for width, precision, or base. In this case, the + fe->size field will indicate how many dots have appeared + in the pattern specification. The `I' requests an integer + to define the object size. + + f: This is the input/output stream in the calling for- + matting function. During a call to fe->extf, the + stream shall be unlocked so that fe->extf can read + from or write to it as appropriate. + + v: For both sfscanf() and sfprintf() functions, v + points to a location suitable for storing any prim- + itive data types, i.e., scalars or pointers. On + return, fe->extf treats v as discussed below. + + fe: This is the current formatting environment. + + The return value rv of fe->extf directs further process- + ing. There are two cases. When pos$ is present, a nega- + tive return value means to ignore fe in further argument + processing while a non-negative return value is treated as + the case rv == 0 below. When pos$ is not present, + fe->extf is called per argument immediately before pattern + processing and its return values are treated as below: + + rv < 0: + The environment stack is immediately popped. + + rv == 0: + The extension function has not consumed (in a scan- + ning case) or output (in a printing case) data out + of or into the given stream f. The fields fmt, + flags, size, width, precis and base of fe shall + direct further processing. + + For sfprintf() functions, if fe->flags has the bit + SFFMT_VALUE, fe->extf should have set *v to the + value to be processed; otherwise, a value should be + obtained from the argument list. Likewise, for + sfscanf() functions, SFFMT_VALUE means that *v + should have a suitable address; otherwise, an + address to assign value should be obtained from the + argument list. + + When pos$ is present, if fe->extf changes fe->fmt, + this pattern shall be used regardless of the pat- + tern defined in the format string. On the other + hand, if fe->fmt is unchanged by fe->extf, the pat- + tern in the format string is used. In any case, + the effective pattern should be one of the + + + + 05 August 1999 17 + + + + + +SFIO(3) SFIO(3) + + + standardly defined pattern. Otherwise, it shall be + treated as unmatched. + + rv > 0: + The extension function has accessed the stream f to + the extent of rv bytes. Processing of the current + pattern ceases except that, for scanning functions, + if fe->flags does not contain the bit SFFMT_SKIP, + the assignment count shall increase by 1. + + + vvooiidd vvaa__ccooppyy((vvaa__lliisstt ttoo,, vvaa__lliisstt ffrr)) + This macro function portably copies the argument list fr + to the argument list to. It should be used to set the + field Sffmt_t.args. + + + lloonngg ssffffmmttvveerrssiioonn((SSffffmmtt__tt** ffee,, iinntt ttyyppee)) + This macro function initializes the formatting environment + fe with a version number if type is non-zero. Otherwise, + it returns the current value of the version number of fe. + This is useful for applications to find out when the for- + mat of the structure Sffmt_t changes. Note that the ver- + sion number corresponds to the Sfio version number which + is defined in the macro value SFIO_VERSION. + + + iinntt ssffpprriinnttff((SSffiioo__tt** ff,, ccoonnsstt cchhaarr** ffoorrmmaatt,, ......)) + cchhaarr** ssffpprriinnttss((ccoonnsstt cchhaarr** ffoorrmmaatt,, ......)) + iinntt ssffsspprriinnttff((cchhaarr** ss,, iinntt nn,, ccoonnsstt cchhaarr** ffoorrmmaatt,, ......)) + iinntt ssffvvsspprriinnttff((cchhaarr** ss,, iinntt nn,, ccoonnsstt cchhaarr** ffoorrmmaatt,, vvaa__lliisstt + aarrggss)) + iinntt ssffvvpprriinnttff((SSffiioo__tt** ff,, ccoonnsstt cchhaarr** ffoorrmmaatt,, vvaa__lliisstt aarrggss)) + These functions format output data. sfprintf() and + sfvprintf() write to output stream f. sfsprintf() and + sfvsprintf() write to buffer s which is of size n. + sfprints() constructs output in some Sfio-defined buffer. + sfvprintf() is the underlying primitive for the other + functions. Except for sfprints() which returns a null- + terminated string or NULL, other functions return the num- + ber of output bytes or -1 on failure. + + The length of string constructed by sfprints(), + sfsprintf(), or sfvsprintf() can be retrieved by sfslen(). + + The standard patterns are: n, s, c, %, h, i, d, p, u, o, + x, X, g, G, e, E, f and !. Except for ! which shall be + described below, see the ANSI-C specification of + fprintf(3) for details on the other patterns. Let z be + some pattern type. A formatting pattern is defined as + below: + + %[pos$][flag][width][.precision][.base][(type)]z + + + + + 05 August 1999 18 + + + + + +SFIO(3) SFIO(3) + + + pos$: A pattern can specify which argument in the argu- + ment list to use. This is done via pos$ where pos + is the argument position. Arguments are numbered + so that the first argument after format is at posi- + tion 1. If pos is not specified, the argument fol- + lowing the most recently used one will be used. + The pattern %! (see below) cannot be used subse- + quent to a usage of pos$. Doing so may cause unex- + pected behaviors. + + flag: The flag characters are h, l, L, I, -, +, _s_p_a_c_e, 0, + ' and #. + + Flag I defines the size or type of the object being + formatted. There are two cases: (1) I by itself + and (2) I followed by either a decimal number or + `*'. + + In the first case, for integer and floating point + patterns, the object type is taken to be the + largest appropriate type (i.e., one of Sflong_t, + Sfulong_t or Sfdouble_t). For conversion speci- + fiers s and c, the flag is ignored. + + In the second case, a given decimal value would + define a size while `*' would cause the size to be + obtained from the argument list. Then, if the con- + version specifier is s, this size defines the + length of the string or strings being formatted + (see the discussion of base below). For integer + and floating point patterns, the size is used to + select a type from one of the below lists as indi- + cated by the conversion specifier: + + Sflong_t, long, int, short + Sfulong_t, unsigned long, unsigned int, unsigned short + Sfdouble_t, double, float + + The selection algorithm always matches types from + left to right in any given list. Although selec- + tion is generally based on sizes in bytes, for com- + patibility with Microsoft-C, the size 64 is matched + with an appropriate type with the same number of + bits, if any. If the given size does not match any + of the listed types, it shall match one of int, + unsigned int, and double as defined by the format- + ting pattern. + + Below are a few examples of using the I flag. The + first example prints an Sflong_t integer. This + example is actually not portable and only works on + platforms where sizeof(Sflong_t) is 8. The second + example shows how to that portably. The third + example specifies printing a string of length 16. + + + + 05 August 1999 19 + + + + + +SFIO(3) SFIO(3) + + + This length shall be used regardless of whether or + not the given string is shorter or longer than 16. + The last example shows the use of the pattern %n to + assign the amount of data already output into a + short integer n_output. + + sfprintf(sfstdout,"%I8d", Sflong_obj); + sfprintf(sfstdout,"%I*d", sizeof(Sflong_obj), Sflong_obj); + sfprintf(sfstdout,"%I*s", 16, s); + sfprintf(sfstdout,"%d%I*n", 1001, sizeof(short), &n_output); + + Flags h, l, and L are the ANSI-C conventions to + select the types of input objects. For example, + %hd indicates a short int while %ld indicates a + long int. The flags ll and L address respectively + the largest integer and floating value types, i.e., + Sfulong_t, Sflong_t, and Sfdouble_t. + + Flag - left-justifies data within the field (other- + wise, right-justification). + + Flag + means that a signed conversion will always + begin with a plus or minus sign. + + Flag _s_p_a_c_e is ignored if + is specified; otherwise, + it means that if the first character of a signed + conversion is not a sign or if the result is empty, + a space will be prepended. + + Flag 0 means padding with zeros on the left. + + Flag ' outputs thousands-separator used by the cur- + rent locale. setlocale(3) should have been used to + set the desired locale. + + Flag # indicates an alternative format processing. + For %o, the first digit is always a zero. For %x + and %X, a non-zero result will have a prefix 0x or + 0X. For %e, %E, %f, %g, and %G, the result always + contains a decimal point. For %g and %G, trailing + zeros will not be removed. For %d, %i and %u, the + form is _b_a_s_e_#_n_u_m_b_e_r where _b_a_s_e is the conversion + base and _n_u_m_b_e_r is represented by digits for this + _b_a_s_e. For example, a base 2 conversion with %#..2d + for 10 is 2#1010 instead of 1010 as printed with + %..2d. + + width: This defines the width of the printing field. A + value to be printed will be justified and padded if + necessary to fill out the field width. + + precis: + After a first dot appears, an integral value + defines a precision. For floating point value + + + + 05 August 1999 20 + + + + + +SFIO(3) SFIO(3) + + + patterns, precision is the number of precision dig- + its. For %c, precision defines the number of times + to repeat the character being formatted. For %s, + precision defines the maximum number of characters + to output. + + base: After a second dot appears, an integral value + defines a conversion base. + + For %i, %d, and %u, a conversion base should be in + the inclusive range [2,64]. If base is not in this + range, it is defined to be 10. The digits to rep- + resent numbers are: + + 01234567890 + abcdefghijklmnopqrstuvwxyz + ABCDEFGHIJKLMNOPQRSTUVWXYZ + @_ + + For %s, when base is defined (i.e., at least two + dots appeared in the pattern specification), the + input argument is taken as a NULL-terminated list + of strings instead of a single string. Each string + is formatted based on the usual width and precision + rules. If base is non-zero, it defines an ASCII + character used to separate the formatted strings. + Below is an example showing both the call and the + result of printing a NULL-terminated list of three + strings apple, orange, and grape: + + sfprintf(sfstdout,"%..*s",',',list); + apple,orange,grape + + Likewise, for %c, when base is defined, the input + argument is taken as a null-terminated string + instead of a single character. Each character is + formatted based on the normal width and precision + rules. In addition, if base is non-zero, it + defines an ASCII character used to separate the + formatted characters if there are more than one. + + (str): This defines a string str to be passed to the + extension function Sffmt_t.extf. Parentheses shall + be balanced. If str is *, the string is obtained + from the argument list. + + + + iinntt ssffssccaannff((SSffiioo__tt** ff,, ccoonnsstt cchhaarr** ffoorrmmaatt,, ......)) + iinntt ssffssssccaannff((ccoonnsstt cchhaarr** ss,, ccoonnsstt cchhaarr** ffoorrmmaatt,, ......)) + iinntt ssffvvssssccaannff((ccoonnsstt cchhaarr** ss,, ccoonnsstt cchhaarr** ffoorrmmaatt,, vvaa__lliisstt + aarrggss)) + + + + + + 05 August 1999 21 + + + + + +SFIO(3) SFIO(3) + + + iinntt ssffvvssccaannff((SSffiioo__tt** ff,, ccoonnsstt cchhaarr** ffoorrmmaatt,, vvaa__lliisstt aarrggss)) + These functions scan data items. sfscanf() scans from the + input stream f while sfsscanf() and sfvsscanf() scan from + the null-terminated string s. sfvscanf() is the underly- + ing primitive that performs the actual scanning. Item + types are determined from patterns in string format. + These functions return the number of items successfully + scanned or -1 on error. + + A white space character (blank, tab, or new-line) in for- + mat normally matches a maximal sequence of input white + space characters. However, if the input stream is in + SF_LINE mode (see sfset()), a new-line character only + matches white spaces up to an input new-line character. + This is useful to avoid blocking when scanning typed + inputs. + + The standard scan patterns are: i, d, u, o, x, X, p, n, f, + e, E, g, G, c, %, s, [] and !. Except for ! which shall + be described below, see the ANSI-C specification of + fscanf(3) for details on other patterns. Let z be some + pattern type. A formatting pattern is specified as below: + + %[*][pos$][width][.width.base][(type)][flag]z + + + pos$: A pattern can specify which argument in the argu- + ment list to use. This is done via pos$ where pos + is the argument position. Arguments are numbered + so that the first argument after format is at posi- + tion 1. If pos is not specified, the argument fol- + lowing the most recently used one will be used. + The pattern %! (see below) cannot be used subse- + quent to a usage of pos$. + + *: This discards the corresponding scanned item. + + width and base: + width defines the maximum number of bytes to scan + and base defines the base of an integral value + being scanned. The `.' (dot) notation also allows + specifying a `*' (star) to obtain the value from + the argument list. The below example specifies + scanning 4 bytes to obtain the value of an integer + in base 10. At the end of scanning, the variable v + should have the value 1234. + + sfsscanf("12345678","%.*.*d", 4, 10, &v); + + + (str): This defines a string str to be passed to the + extension function Sffmt_t.extf. Parentheses shall + be balanced. If str is *, the string is obtained + from the argument list. + + + + 05 August 1999 22 + + + + + +SFIO(3) SFIO(3) + + + flag: This is #, I, or some sequence of h, l, and L. + + Flag # is significant for pattern %i and %[. For + %i, it means that the # symbol does not have its + usual meaning in an input sequence base#value. For + example, the scanning result of %#i on input 2#1001 + is 2 and the next sfgetc() call will return #. For + %[, if the next character in the input stream does + not match the given scan set of characters, # + causes a match to a null string instead of a fail- + ure. + + Flag I defines the size or type of the object being + formatted. There are two cases: (1) I by itself + and (2) I followed by either a decimal number or + `*'. + + In the first case, for integer and floating point + patterns, the object type is taken to be the + largest appropriate type (i.e., one of Sflong_t, + Sfulong_t or Sfdouble_t). For string patterns such + as %s, the flag is ignored. + + In the second case, a given decimal value would + define a size while `*' would cause the size to be + obtained from the argument list. For string pat- + terns such as %s or %[, this size defines the + length of the buffer to store scanned data. Speci- + fying a buffer size only limits the amount of data + copied into the buffer. Scanned data beyond the + buffer limit will be discarded. For integer and + floating point patterns, the size is used to select + a type from one of the below lists as indicated by + the conversion specifier: + + Sflong_t, long, int, short + Sfulong_t, unsigned long, unsigned int, unsigned short + Sfdouble_t, double, float + + The selection algorithm always matches types from + left to right in any given list. Although selec- + tion is generally based on sizes in bytes, for com- + patibility with Microsoft-C, the size 64 is matched + with an appropriate type with the same number of + bits, if any. If the given size does not match any + of the listed types, it shall match one of int, + unsigned int, and double as indicated by the for- + matting pattern. + + Below are examples of using the I flag. The first + example scans a 64-bit integer. The second scans + some floating point value whose size is explicitly + computed and given. The last example scans a + string into a buffer with the given size 128. Note + + + + 05 August 1999 23 + + + + + +SFIO(3) SFIO(3) + + + that if the scanned string is longer than 127, only + the first 127 bytes shall be copied into the + buffer. The rest of the scanned data shall be dis- + carded. + + sfscanf(sfstdin,"%I64d", &int64_obj); + sfscanf(sfstdin,"%I*f", sizeof(float_obj), &float_obj); + sfscanf(sfstdin,"%I*s", 128, buffer); + + Flags h, l, and L are the ANSI-C conventions for + indicating the type of a scanned element. For + example, %hd means scanning a short int. The flags + ll and L mean respectively scanning an integer or a + floating point value with largest size (i.e, + Sflong_t or Sfdouble_t). + + The %i, %d and %u patterns scan numbers in bases from 2 to + 64. %i scans integral values in self-describing formats. + Except for octal, decimal and hexadecimal numbers with the + usual formats, numbers in general bases are assumed to be + of the form: _b_a_s_e_#_v_a_l_u_e where _b_a_s_e is a number in base 10 + and _v_a_l_u_e is a number in the given base. For example, + 2#1001 is the binary representation of the decimal value + 9. If _b_a_s_e is 36 or less, the digits for _v_a_l_u_e can be any + combination of [0-9], [a-z], [A-Z] where upper and lower + case digits are not distinguishable. If _b_a_s_e is larger + than 36, the set of digits is: + + 0123456789 + abcdefghijklmnopqrstuvwxyz + ABCDEFGHIJKLMNOPQRSTUVWXYZ + @_ + + + + BBUUFFFFEERRIINNGG,, SSYYNNCCHHRROONNIIZZAATTIIOONN + VVooiidd__tt** ssffsseettbbuuff((SSffiioo__tt** ff,, VVooiidd__tt** bbuuff,, ssiizzee__tt ssiizzee)) + This function sets the buffering scheme for the stream f. + Except for buffer inquiry (see the case size == 0,) f will + be synchronized before any buffer modification. If a new + buffer is successfully set and the old buffer has not been + deallocated, sfsetbuf() returns the address of the old + buffer. Otherwise, it returns NULL. + + After a sfsetbuf() call, sfvalue() returns the size of the + returned buffer. + + size == SF_UNBOUND: + Sfio will pick a suitable buffer size. If buf is + NULL, Sfio will also pick a suitable buffering + scheme (such as memory mapping.) If buf is not + NULL, its actual value is ignored but the buffer + will be allocated via malloc(3). This can be used + to avoid memory mapping. + + + + 05 August 1999 24 + + + + + +SFIO(3) SFIO(3) + + + size > 0: + This is the suggested size to use for buffering or + memory mapping. If buf is NULL, Sfio will pick a + suitable buffering scheme as discussed above. If + buf is not NULL, then buf and size determine a + buffer of the given size. + + size == 0: + If buf is NULL, the stream will be unbuffered. If + buf is not NULL, sfsetbuf() simply returns the + stream buffer. In this case, no attempt will be + made to synchronize the stream. + + + iinntt ssffssyynncc((SSffiioo__tt** ff)) + This function synchronizes the logical and physical views + of stream f. For a SF_WRITE stream, this means to write + out any buffered data. For a seekable SF_READ file + stream, the physical file position is aligned with the + logical stream position and, if SF_SHARE is on, buffered + data is discarded. If f is NULL, all streams are synchro- + nized. If f is the base of a stream stack (see sfs- + tack()), all stacked streams are synchronized. Note that + a stacked stream can only be synchronized this way. If f + is in a pool (see sfpool()) but not being the head, the + pool head is synchronized. After a successful synchro- + nization, if f has flag SF_IOCHECK, a SF_SYNC event is + raised. sfsync() returns a negative value for failure and + 0 for success. + + + iinntt ssffppoollll((SSffiioo__tt**** fflliisstt,, iinntt nn,, iinntt ttiimmeeoouutt)) + This function polls a set of streams to see if I/O opera- + tions can be performed on them without blocking. This is + useful for multiplexing I/O over a set of streams. If a + stream has a discipline, the exception function may be + called before and after the stream is polled (see sfdisc() + for details). sfpoll() returns the number of ready + streams or -1 on failure. + + flist and n: + flist is an array of n streams to be polled. Upon + return, ready streams are moved to the front of + flist in the same relative order. + + timeout: + This defines an ellapse time in milliseconds to + wait to see if any stream is ready for I/O. If + timeout is negative, sfpoll() will block until some + stream become ready. Note that SF_STRING and nor- + mal file streams never block and are always ready + for I/O. If a stream with discipline is being + polled and its readiness is as yet undetermined + (e.g., empty buffer,) the discipline exception + + + + 05 August 1999 25 + + + + + +SFIO(3) SFIO(3) + + + function will be called with SF_DPOLL before normal + actions are taken. + + + SSffiioo__tt** ssffppooooll((SSffiioo__tt** ff,, SSffiioo__tt** ppoooollff,, iinntt mmooddee)) + This function manipulates pools of streams. In a pool, + only one stream is at the head and can have buffered data. + All other streams in the pool will be synchronized. A + stream becomes head when it is used for some I/O opera- + tion. sfpool() returns NULL on failure. + + f and poolf: + If f is NULL, sfpool() simply returns the head of + the pool containing poolf. If f is not NULL and + poolf is NULL, f is deleted from its pool. In this + case, if no other stream from the same pool can + become head, sfpool() will return NULL; otherwise, + it returns some stream from the remainder of the + pool. If both f and poolf are not NULL, f is moved + from its current pool (if any) into the same pool + with poolf. In this case, poolf is returned. + + mode: If poolf is already in a pool, mode is ignored. + Otherwise, mode should be 0 or SF_SHARE. A + SF_SHARE pool contains streams with SF_WRITE mode. + In addition, on change to a new head stream, + buffered write data of the current head is trans- + ferred to the new head. + + + iinntt ssffppuurrggee((SSffiioo__tt** ff)) + This function discards all buffered data unless f is a + SF_STRING stream. Note that if f is a SF_READ stream + based on an unseekable device, purged data will not be + recoverable. If f is a sfpopen-stream opened for both + read and write, data of both the read and write pipe ends + will be purged (see sfset() to selectively turn off read + or write mode if one set of data is to be preserved.) + After purging, if f has flag SF_IOCHECK, the event + SF_PURGE is raised. sfpurge() returns -1 for failure and + 0 for success. + + + + DDIISSCCIIPPLLIINNEE,, EEVVEENNTT--HHAANNDDLLIINNGG + A file stream uses the system calls read(2), write(2) and + lseek(2) to read, write and position in the underlying + file. Disciplines enable application-defined I/O methods + including exception handling and data pre/post-processing. + + + SSffddiisscc__tt** ssffddiisscc((SSffiioo__tt** ff,, SSffddiisscc__tt** ddiisscc)) + Each stream has a discipline stack whose bottom is a vir- + tual discipline representing the actual system calls. + + + + 05 August 1999 26 + + + + + +SFIO(3) SFIO(3) + + + sfdisc() manipulates the discipline stack of stream f. f + will be synchronized before any discipline stack manipula- + tion. After a successful discipline stack manipulation, + the stream I/O position (see sfseek() and sftell()) and + extent (see sfsize()) are updated to reflect that defined + by the top discipline. If disc is SF_POPDISC or + (Sfdisc_t*)0, the top element of the stack, if any, is + popped and its address is returned. Otherwise, disc is + pushed onto the discipline stack. In this case, if suc- + cessful, sfdisc() returns the discipline that was pushed + down. sfdisc() returns NULL on failure. + + Note that a discipline can be used only on one stream at a + time. An application should take care to allocate differ- + ent discipline structures for use with different streams. + A discipline structure is of the type Sfdisc_t which con- + tains the following public fields: + + Sfread_f readf; + Sfwrite_f writef; + Sfseek_f seekf; + Sfexcept_f exceptf; + + + The first three fields of Sfdisc_t specify alternative I/O + functions. If any of them is NULL, it is inherited from a + discipline pushed earlier on the stack. Note that a file + stream always has read(2), write(2), lseek(2) and + NIL(Sfexcept_f) as the _l_o_g_i_c_a_l _b_o_t_t_o_m _d_i_s_c_i_p_l_i_n_e. Argu- + ments to I/O discipline functions have the same meaning as + that of the functions sfrd(), sfwr() and sfsk() described + below. + + The exception function, (*exceptf)() announces exceptional + events during I/O operations. It is called as + (*exceptf)(Sfio_t* f, int type, Void_t* value, Sfdisc_t* + disc). Unless noted otherwise, the return value of + (*exceptf)() is used as follows: + + <0: The on-going operation shall terminate. + + >0: If the event was raised due to an I/O error, the + error has been repaired and the on-going operation + shall continue normally. + + =0: The on-going operation performs default actions + with respect to the raised event. For example, on + a reading error or reaching end of file, the top + stream of a stack will be popped and closed and the + on-going operation continue with the new top + stream. + + The argument type of (*exceptf)() identifies the particu- + lar exceptional event: + + + + 05 August 1999 27 + + + + + +SFIO(3) SFIO(3) + + + SF_LOCKED: + The stream is in a locked state. + + SF_READ, SF_WRITE: + These events are raised around reading and writing + operations. + + If SF_IOCHECK is on, SF_READ and SF_WRITE are + raised immediately before read(2) and write(2) + calls. In this case, *((ssize_t*)value) is the + amount of data to be processed. The return value + of (*exceptf)(), if negative, indicates that the + stream is not ready for I/O and the calling opera- + tion will abort with failure. If it is positive, + the stream is ready for I/O but the amount should + be restricted to the amount specified by this + value. If the return value is zero, the I/O opera- + tion is carried out normally. + + SF_READ and SF_WRITE are also raised on operation + failures. In such a case, *((ssize_t*)value) is + the return value from the failed operation. + + SF_SEEK: + This event is raised when a seek operation fails. + + SF_NEW, SF_CLOSE, SF_FINAL: + These events are raised during a stream closing. + SF_NEW is raised for a stream about to be closed to + be renewed (see sfnew()). SF_CLOSE is raised for a + stream about to be closed. SF_FINAL is raised + after a stream has been closed and before its space + is to be destroyed (see sfclose()). For these + events, a non-zero return value from (*exceptf)() + causes sfclose() to return immediately with the + same value. + + SF_DPUSH, SF_DPOP, SF_DBUFFER: + Events SF_DPUSH and SF_DPOP are raised when a dis- + cipline is about to be pushed or popped. + (Sfdisc_t*)value is the to-be top discipline, if + any. + + A stream buffer is always synchronized before push- + ing or popping a discipline. If this synchroniza- + tion fails, SF_DBUFFER will be raised with + *((size_t*)value) being the amount of buffered + data. If the return value of exceptf is positive, + the push or pop operation will continue normally; + otherwise, sfdisc() returns failure. + + SF_DPOLL: + This event is raised by sfpoll() to see if the + stream is ready for I/O. *((int*)value) indicates + + + + 05 August 1999 28 + + + + + +SFIO(3) SFIO(3) + + + a time-out interval to wait. A negative return + value from the exception function means blocking. + A positive return value means non-blocking. A zero + return value means that sfpoll() should query the + stream file descriptor using default methods. + + SF_READY: + This event is raised by sfpoll() for each ready + stream after they are determined. A negative + return value from the exeption handler causes + sfpoll() to return immediately with the same return + value. A positive return value causes sfpoll() to + retry polling the whole set of streams. + + SF_SYNC, SF_PURGE: + If SF_IOCHECK is set, these events are raised imme- + diately after sfsync() or sfpurge() successfully + complete their operations and before they return. + Note that sfsync() is implied when a SF_WRITE or + SF_SHARE|SF_READ stream is closed. Note also that + SF_SYNC is not raised for a stream synchronized + during a call sfsync((Sfio_t*)0). + + SF_ATEXIT: + This event is raised for each open stream before + the process exits. + + + iinntt ssffrraaiissee((SSffiioo__tt** ff,, iinntt ttyyppee,, VVooiidd__tt** ddaattaa)) + This function calls all exception handlers of stream f + with the event type and associated data. If an exception + handler returns a non-zero value, sfraise() immediate + returns the same value. Application-defined events should + start from the value SF_EVENT so as to avoid confusion + with system-defined events, sfraise() returns 0 on success + and -1 on failure. + + + ssssiizzee__tt ssffrrdd((SSffiioo__tt** ff,, VVooiidd__tt** bbuuff,, ssiizzee__tt nn,, SSffddiisscc__tt** + ddiisscc)) + ssssiizzee__tt ssffwwrr((SSffiioo__tt** ff,, ccoonnsstt VVooiidd__tt** bbuuff,, ssiizzee__tt nn,, + SSffddiisscc__tt** ddiisscc)) + SSffooffff__tt ssffsskk((SSffiioo__tt** ff,, SSffooffff__tt ooffffsseett,, iinntt ttyyppee,, SSffddiisscc__tt** + ddiisscc)) + These functions provides safe methods for a discipline I/O + function to invoke earlier discipline I/O functions and to + properly handle exceptions. They should not be used in + any other context. sfrd() and sfwr() return the number of + bytes read or written. sfsk() returns the new seek posi- + tion. On error, all three functions return a negative + value which should be -1 or the value returned by the + exception handler. + + + + + + 05 August 1999 29 + + + + + +SFIO(3) SFIO(3) + + + SSTTRREEAAMM CCOONNTTRROOLL + iinntt ssffsseett((SSffiioo__tt** ff,, iinntt ffllaaggss,, iinntt sseett)) + This function sets control flags for the stream f. It + returns the previous set of flags or 0 on error. + + Settable flags are: SF_READ, SF_WRITE, SF_IOCHECK, + SF_LINE, SF_SHARE, SF_PUBLIC, SF_MALLOC, SF_STATIC and + SF_BUFCONST. Note that SF_READ and SF_WRITE can be turned + on or off only if the stream was opened as + SF_READ|SF_WRITE. Turning off one of them means that the + stream is to be treated exclusively in the other mode. It + is not possible to turn off both. If legal, an attempt to + turn on either SF_READ or SF_WRITE will cause the stream + to be in the given I/O mode. + + set == 0: + If flags is zero, the current set of flags is sim- + ply returned. Note that when a stream is first + opened, not all of its flags are initialized yet + (more below). If flags is non-zero, an attempt is + made to turn off the specified flags. + + set != 0: + If flags is zero, the stream is initialized if not + yet done so. Then the current set of flags is + returned. If flags is non-zero, an attempt is made + to turn on the specified flags. + + + iinntt ssffsseettffdd((SSffiioo__tt** ff,, iinntt ffdd)) + This function changes the file descriptor of f. Before a + change is realized, (*notify)(f,SF_SETFD,newfd) (see sfno- + tify()) is called. sfsetfd() returns -1 on failure and + the new file descriptor on success. + + fd >= 0: + If the current file descriptor is non-negative, it + will be changed using dup(3) to a value larger or + equal to fd. Upon a successful change, the previ- + ous file descriptor will be closed. If the current + file descriptor is negative, it will be set to fd + and the stream will be reinitialized. + + fd < 0: + The stream is synchronized (see sfsync()) and its + file descriptor will be set to this value. Then, + except for sfclose(), the stream will be inaccessi- + ble until a future sfsetfd() call resets the file + descriptor to a non-negative value. Thus, + sfsetfd(f,-1) can be used to avoid closing the file + descriptor of f when f is closed. + + + + + + + 05 August 1999 30 + + + + + +SFIO(3) SFIO(3) + + + SSffiioo__tt** ssffssttaacckk((SSffiioo__tt** bbaassee,, SSffiioo__tt** ttoopp)) + This function stacks or unstacks stream. Every stream + stack is identified by a base stream via which all I/O + operations are performed. However, an I/O operation + always takes effect on the top stream. If the top stream + reaches the end of file or has an unrecoverable error con- + dition, it is automatically popped and closed (see also + sfdisc() for alternative handling of these conditions). + + base: This is the base stream of the stack. If it is + NULL, sfstack() does nothing and returns top. + + top: If this is SF_POPSTACK or (Sfio_t*)0, the stack is + popped and sfstack() returns the popped stream. + Otherwise, top is pushed on top of the stack iden- + tified by base and sfstack() returns the base + stream. + + + SSffiioo__tt** ssffsswwaapp((SSffiioo__tt** ff11,, SSffiioo__tt** ff22)) + This function swaps contents of f1 and f2. This fails if + either stream is in a stream stack but not being a base + stream. If f2 is NULL, a new stream is constructed as a + duplicate of f1. sfswap() returns f2 or f1 duplicate on + success and NULL on failure. + + + + SSTTRREEAAMM IINNFFOORRMMAATTIIOONN + SSffooffff__tt ssffssiizzee((SSffiioo__tt** ff)) + This function returns the size of stream f (see sfnew()). + If f is not seekable or if its size is not determinable, + sfsize() returns -1. + + + SSffooffff__tt ssfftteellll((SSffiioo__tt** ff)) + This function returns the current I/O position in stream + f. Note that if f is SF_APPEND and a writing operation + was just performed, the current I/O position is at the + physical end of file. If f is unseekable, sftell returns + the number of bytes read from or written to f. See also + sfungetc(). + + + ssssiizzee__tt ssffvvaalluuee((SSffiioo__tt** ff)) + This function returns the string or buffer length for + sfreserve(), sfsetbuf(), and sfgetr(). + + + iinntt ssffffiilleennoo((SSffiioo__tt** ff)) + This function returns the file descriptor of stream f. + + + + + + + 05 August 1999 31 + + + + + +SFIO(3) SFIO(3) + + + iinntt ssffssttaacckkeedd((SSffiioo__tt** ff)) + This function returns a non-zero value if stream f has + been stacked. + + + iinntt ssffeeooff((SSffiioo__tt** ff)) + iinntt ssffeerrrroorr((SSffiioo__tt** ff)) + iinntt ssffccllrreerrrr((SSffiioo__tt** ff)) + sfeof() tells whether or not the stream has an end-of-file + condition. sferror() tells whether or not the stream has + an error condition. sfclrerr() clears both end-of-file + and error conditions. The end-of-file and error condi- + tions are also cleared on an I/O operation. + + + iinntt ssffccllrrlloocckk((SSffiioo__tt** ff)) + This function restores the stream back to a normal state. + This means clearing locks and possibly throwing away + unprocessed data. As such, this operation is unsafe and + should be used with care. For example, it may be used + before a long jump (longjmp(3)) out of some discipline I/O + function to restore the internal stream states. sfclr- + lock() returns the current set of flags. + + + iinntt ssffnnoottiiffyy((((vvooiidd((**))nnoottiiffyy))((SSffiioo__tt**,, iinntt,, iinntt)) )) + This sets a function (*notify)() to be called as + (*notify)(f,type,file) on various stream events. Argu- + ments f and file are stream and related file descriptor. + Argument type indicates the reason for the call: + + SF_NEW: + f is being opened and file is the underlying file + descriptor. + + SF_CLOSE: + f and file are being closed. + + SF_SETFD: + The file descriptor of f is being changed to file + (see sfsetfd().) + + SF_READ: + An attempt to change f to read mode failed. + + SF_WRITE: + An attempt to change f to write mode failed. + + + + MMIISSCCEELLLLAANNEEOOUUSS FFUUNNCCTTIIOONNSS + ssssiizzee__tt ssffsslleenn(()) + This function returns the length of a string just con- + structed by sfsprintf() or sfprints(). See also + + + + 05 August 1999 32 + + + + + +SFIO(3) SFIO(3) + + + sfvalue(). + + + iinntt ssffuulleenn((SSffuulloonngg__tt vv)) + iinntt ssfflllleenn((SSfflloonngg__tt vv)) + iinntt ssffddlleenn((SSffddoouubbllee__tt vv)) + These functions return respectively the number of bytes + required to code the Sfulong_t, Sflong_t or Sfdouble_t + value v by sfputu(), sfputl() or sfputd(). + + + ssssiizzee__tt ssffppkkrrdd((iinntt ffdd,, cchhaarr** bbuuff,, ssiizzee__tt nn,, iinntt rrsscc,, lloonngg + ttmm,, iinntt aaccttiioonn)) + This function acts directly on the file descriptor fd. It + does a combination of peeking on incoming data and a time- + out read. Upon success, it returns the number of bytes + received. A return value of 0 means that the end-of-file + condition has been detected. A negative value represents + an error. + + buf, n: + These define a buffer and its size to read data + into. + + rsc: If >=0, this defines a record separator. See + action for detail. + + tm: If >=0, this defines a time interval in millisec- + onds to wait for incoming data. + + action: + When rsc >= 0, the absolute value of action, _r, + determines the number of records to be read. If + action > 0, sfpkrd() will peek on incoming data but + will not read past it. Therefore, a future sfpkrd() + or read(2) will see the same data again. If action + == 0, sfpkrd() will not peek. If action < 0, there + are two cases: if rsc < 0, sfpkrd() reads n bytes; + otherwise, exactly _r records will be read. Note + that, in the last case, reading records from an + unseekable device may be slow if the underlying + platform does not allow peeking on such a device. + + + + FFUULLLL SSTTRRUUCCTTUURREE SSFFIIOO__TT + ##iinncclluuddee <> + Most applications based on Sfio only need to include the + header file sfio.h which defines an abbreviated Sfio_t + structure without certain fields private to Sfio. How- + ever, there are times (e.g., debugging) when an applica- + tion may require more details about the full Sfio_t struc- + ture. In such cases, the header file sfio_t.h can be used + in place of sfio.h. Note that an application doing this + + + + 05 August 1999 33 + + + + + +SFIO(3) SFIO(3) + + + will become sensitive to changes in the internal architec- + ture of Sfio. + + + ##ddeeffiinnee SSFFNNEEWW((bbuuff,,ssiizzee,,ffiillee,,ffllaaggss,,ddiisscc)) + This macro function is defined in sfio_t.h for use in + static initialization of an Sfio_t structure. It requires + five arguments: + + buf, size: + These define a buffer and its size. + + file: This defines the underlying file descriptor if any. + + flags: This is composed from bit flags described above. + + disc: This defines a discipline if any. + + + + EEXXAAMMPPLLEE DDIISSCCIIPPLLIINNEESS + The below functions create disciplines and insert them + into the given streams f. These functions return 0 on suc- + cess and -1 on failure. + + + iinntt ssffddccddiioo((SSffiioo__tt** ff,, ssiizzee__tt bbuuffssiizzee)) + This creates a discipline that uses the direct IO feature + available on file systems such as SGI's XFS to speed up + IO. The argument bufsize suggests a buffer size to use + for data transfer. + + + iinntt ssffddccddooss((SSffiioo__tt** ff)) + This creates a discipline to read DOS text files. It + basically transforms pairs of \r\n to \n. + + + iinntt ssffddccffiilltteerr((SSffiioo__tt** ff,, ccoonnsstt cchhaarr** ccmmdd)) + This creates a discipline that sends data from f to the + given command cmd to process, then reads back the pro- + cessed data. + + + iinntt ssffddccllzzww((SSffiioo__tt** ff)) + This creates a discipline that would decompress data in f. + The stream f should have data from a source compressed by + the Unix ccoommpprreessss program. + + + iinntt ssffddccsseeeekkaabbllee((SSffiioo__tt** ff)) + This creates a discipline that makes an unseekable reading + stream seekable. + + + + + 05 August 1999 34 + + + + + +SFIO(3) SFIO(3) + + + iinntt ssffddccssllooww((SSffiioo__tt** ff)) + This creates a discipline that makes all Sfio operations + return immediately on interrupts. This is useful for deal- + ing with slow devices. + + + iinntt ssffddccssuubbssttrreeaamm((SSffiioo__tt** ff,, SSffiioo__tt** ppaarreenntt,, SSffooffff__tt ooffffsseett,, + SSffooffff__tt eexxtteenntt)) + This creates a discipline that makes f acts as if it cor- + responds exactly to the subsection of parent starting at + offset with size extent. + + + iinntt ssffddcctteeee((SSffiioo__tt** ff,, SSffiioo__tt** tteeee)) + This creates a discipline that copies to the stream tee + any data written to f. + + + iinntt ssffddccuunniioonn((SSffiioo__tt** ff,, SSffiioo__tt**** aarrrraayy,, iinntt nn)) + This creates a discipline that makes f act as if it is the + concatenation of the n streams given in array. + + + + SSTTDDIIOO--CCOOMMPPAATTIIBBIILLIITTYY + Sfio provides two compatibility packages to Stdio-based + applications, a source level interface and a binary level + library. These packages provide a union of functions in + popular Stdio implementations. + + The source Stdio-compatibility interface provides the + header file stdio.h that defines a set of macros or + inlined functions to map Stdio calls to Sfio ones. This + mapping may benignly extend or change the meaning of cer- + tain original Stdio operations. For example, the Sfio's + version of popen() allows a coprocess to be opened for + both reading and writing unlike the original call which + only allows a coprocess to be opened for a single mode. + Similarly, the Sfio's fopen() call can be used to create + string streams in addition to file streams. + + The binary Stdio-compatibility library, libstdio.a, pro- + vides a complete implementation of Stdio functions suit- + able for linking applications already compiled with native + header stdio.h. Functions in this implementation are also + slightly altered or extended as discussed above. + + Below are the supported Stdio functions: + + FILE* fopen(const char* file, const char* mode); + FILE* freopen(const char* file, const char* mode, FILE* stream); + FILE* fdopen(int filedesc, const char* mode); + FILE* popen(const char* command, const char* mode); + FILE* tmpfile(); + + + + 05 August 1999 35 + + + + + +SFIO(3) SFIO(3) + + + int fclose(FILE* stream); + int pclose(FILE* stream); + + void setbuf(FILE* stream, char* buf); + int setvbuf(FILE* stream, char* buf, int mode, size_t size); + void setbuffer(FILE* stream, char* buf, size_t size); + int setlinebuf(FILE* stream); + int fflush(FILE* stream); + int fpurge(FILE* stream); + + int fseek(FILE* stream, long offset, int whence); + void rewind(FILE* stream); + int fgetpos(FILE* stream, fpos_t* pos); + int fsetpos(FILE* stream, fpos_t* pos); + long ftell(FILE* stream); + + int getc(FILE* stream); + int fgetc(FILE* stream); + int getchar(void); + int ungetc(int c, FILE* stream); + int getw(FILE* stream); + char* gets(char* s); + char* fgets(char* s, int n, FILE* stream); + size_t fread(Void_t* ptr, size_t size, size_t nelt, FILE* stream); + + int putc(int c, FILE* stream); + int fputc(int c, FILE* stream); + int putchar(int c); + int putw(int w, FILE* stream); + int puts(const char* s, FILE* stream); + int fputs(const char* s, FILE* stream); + size_t fwrite(const Void_t* ptr, size_t size, size_t nelt, FILE* stream); + + int fscanf(FILE* stream, const char* format, ...); + int vfscanf(FILE* stream, const char* format, va_list args); + int _doscan(FILE* stream, const char* format, va_list args); + int scanf(const char* format, ...); + int vscanf(const char* format, va_list args); + int sscanf(const char* s, const char* format, ...); + int vsscanf(const char* s, const char* format, va_list args); + + int fprintf(FILE* stream, const char* format, ...); + int vfprintf(FILE* stream, const char* format, va_list args); + int _doprnt(FILE* stream, const char* format, va_list args); + int printf(const char* format, ...); + int vprintf(const char* format, va_list args); + int sprintf(const char* s, const char* format, ...); + int snprintf(const char* s, int n, const char* format, ...); + int vsprintf(const char* s, const char* format, va_list args); + int vsnprintf(const char* s, int n, const char* format, va_list args); + + int feof(FILE* stream); + int ferror(FILE* stream); + int clearerr(FILE* stream); + + + + 05 August 1999 36 + + + + + +SFIO(3) SFIO(3) + + + RREECCEENNTT CCHHAANNGGEESS + A few exception types have been added. In particular, + exception handlers shall be raised with SF_LOCKED on + accessing a locked stream. Before a process exits, the + event SF_ATEXIT is raised for each open stream. + + A number of disciplines have been added for various pro- + cessing functions. Of interests are disciplines to use + the direct I/O feature on IRIX6.2, read DOS text files, + and decompress files compressed by Unix _c_o_m_p_r_e_s_s. + + Various new stream and function flags have been added. For + example, the third argument of sfgetr() is now a set of + bit flags and not just a three-value object. However, the + old semantics of this argument of sfgetr() is still sup- + ported. + + The sfopen() call has been extended so that + sfopen(f,NULL,mode) can be used to changed the mode of a + file stream before any I/O operations. This is most use- + ful for changing the modes of the standard streams. + + The buffering strategy has been significantly enhanced for + streams that perform many seek operations. Also, the han- + dling of stream and file positions have been better clari- + fied so that applications that share file descriptors + across streams and/or processes can be sure that the file + states will be consistent. + + +AAUUTTHHOORRSS + Kiem-Phong Vo, kpv@research.att.com, + David G. Korn, dgk@research.att.com, and + Glenn S. Fowler, gsf@research.att.com. + + + + + + + + + + + + + + + + + + + + + + + + 05 August 1999 37 + + Index: ossp-pkg/sio/BRAINSTORM/bsd-stdio.txt RCS File: /v/ossp/cvs/ossp-pkg/sio/BRAINSTORM/bsd-stdio.txt,v co -q -kk -p'1.1' '/v/ossp/cvs/ossp-pkg/sio/BRAINSTORM/bsd-stdio.txt,v' | diff -u /dev/null - -L'ossp-pkg/sio/BRAINSTORM/bsd-stdio.txt' 2>/dev/null --- ossp-pkg/sio/BRAINSTORM/bsd-stdio.txt +++ - 2024-05-15 09:15:08.635636005 +0200 @@ -0,0 +1,165 @@ + +STDIO(3) FreeBSD Library Functions Manual STDIO(3) + +NAME + stdio - standard input/output library functions + +SYNOPSIS + #include + FILE *stdin; + FILE *stdout; + FILE *stderr; + +DESCRIPTION + The standard I/O library provides a simple and efficient buffered stream + I/O interface. Input and output is mapped into logical data streams and + the physical I/O characteristics are concealed. The functions and macros + are listed below; more information is available from the individual man + pages. + + A stream is associated with an external file (which may be a physical de- + vice) by opening a file, which may involve creating a new file. Creating + an existing file causes its former contents to be discarded. If a file + can support positioning requests (such as a disk file, as opposed to a + terminal) then a file position indicator associated with the stream is + positioned at the start of the file (byte zero), unless the file is + opened with append mode. If append mode is used, the position indicator + will be placed at the end-of-file. The position indicator is maintained + by subsequent reads, writes and positioning requests. All input occurs as + if the characters were read by successive calls to the fgetc(3) function; + all output takes place as if all characters were written by successive + calls to the fputc(3) function. + + A file is disassociated from a stream by closing the file. Output + streams are flushed (any unwritten buffer contents are transferred to the + host environment) before the stream is disassociated from the file. The + value of a pointer to a FILE object is indeterminate (garbage) after a + file is closed. + + A file may be subsequently reopened, by the same or another program exe- + cution, and its contents reclaimed or modified (if it can be repositioned + at the start). If the main function returns to its original caller, or + the exit(3) function is called, all open files are closed (hence all out- + put streams are flushed) before program termination. Other methods of + program termination may not close files properly and hence buffered out- + put may be lost. In particular, _exit(2) does not flush stdio files. + Neither does an exit due to a signal. Buffers are flushed by abort(3) as + required by POSIX, although previous implementations did not. + + This implementation makes no distinction between ``text'' and ``binary'' + streams. In effect, all streams are binary. No translation is performed + and no extra padding appears on any stream. + + At program startup, three streams are predefined and need not be opened + explicitly: + o standard input (for reading conventional input), + o standard output (for writing conventional output), and + o standard error (for writing diagnostic output). + These streams are abbreviated stdin, stdout and stderr. Initially, the + standard error stream is unbuffered; the standard input and output + streams are fully buffered if and only if the streams do not refer to an + interactive or ``terminal'' device, as determined by the isatty(3) func- + tion. In fact, all freshly-opened streams that refer to terminal devices + default to line buffering, and pending output to such streams is written + automatically whenever such an input stream is read. Note that this ap- + plies only to ``true reads''; if the read request can be satisfied by ex- + isting buffered data, no automatic flush will occur. In these cases, or + when a large amount of computation is done after printing part of a line + on an output terminal, it is necessary to fflush(3) the standard output + before going off and computing so that the output will appear. Alterna- + tively, these defaults may be modified via the setvbuf(3) function. + + The stdio library is a part of the library libc and routines are automat- + ically loaded as needed by the C compiler. The SYNOPSIS sections of the + following manual pages indicate which include files are to be used, what + the compiler declaration for the function looks like and which external + variables are of interest. + + The following are defined as macros; these names may not be re-used with- + out first removing their current definitions with #undef: BUFSIZ, EOF, + FILENAME_MAX, FOPEN_MAX, L_cuserid, L_ctermid, L_tmpnam, NULL, P_tmpdir, + SEEK_CUR, SEEK_END, SEEK_SET, TMP_MAX, clearerr, feof, ferror, fileno, + fropen, fwopen, getc, getchar, putc, putchar, stderr, stdin, stdout, + vfscanf. Function versions of the macro functions clearerr(), feof(), + ferror(), fileno(), getc(), getchar(), putc(), and putchar() exist and + will be used if the macro definitions are explicitly removed. + +SEE ALSO + close(2), open(2), read(2), write(2) + +BUGS + The standard buffered functions do not interact well with certain other + library and system functions, especially vfork(2). + +STANDARDS + The stdio library conforms to ISO 9899: 1990 (``ISO C''). + +LIST OF FUNCTIONS + Function Description + asprintf formatted output conversion + clearerr check and reset stream status + fclose close a stream + fdopen stream open functions + feof check and reset stream status + ferror check and reset stream status + fflush flush a stream + fgetc get next character or word from input stream + fgetln get a line from a stream + fgetpos reposition a stream + fgets get a line from a stream + fileno check and reset stream status + fopen stream open functions + fprintf formatted output conversion + fpurge flush a stream + fputc output a character or word to a stream + fputs output a line to a stream + fread binary stream input/output + freopen stream open functions + fropen open a stream + fscanf input format conversion + fseek reposition a stream + fsetpos reposition a stream + ftell reposition a stream + funopen open a stream + fwopen open a stream + fwrite binary stream input/output + getc get next character or word from input stream + getchar get next character or word from input stream + gets get a line from a stream + getw get next character or word from input stream + mkdtemp create unique temporary file + mkstemp create unique temporary file + mktemp create unique temporary file + perror system error messages + printf formatted output conversion + putc output a character or word to a stream + putchar output a character or word to a stream + puts output a line to a stream + putw output a character or word to a stream + remove remove directory entry + rewind reposition a stream + scanf input format conversion + setbuf stream buffering operations + setbuffer stream buffering operations + setlinebuf stream buffering operations + setvbuf stream buffering operations + snprintf formatted output conversion + sprintf formatted output conversion + sscanf input format conversion + strerror system error messages + sys_errlist system error messages + sys_nerr system error messages + tempnam temporary file routines + tmpfile temporary file routines + tmpnam temporary file routines + ungetc un-get character from input stream + vasprintf formatted output conversion + vfprintf formatted output conversion + vfscanf input format conversion + vprintf formatted output conversion + vscanf input format conversion + vsnprintf formatted output conversion + vsprintf formatted output conversion + vsscanf input format conversion + +4th Berkeley Distribution April 19, 1994 3 Index: ossp-pkg/sio/BRAINSTORM/openssl-bio.txt RCS File: /v/ossp/cvs/ossp-pkg/sio/BRAINSTORM/openssl-bio.txt,v co -q -kk -p'1.1' '/v/ossp/cvs/ossp-pkg/sio/BRAINSTORM/openssl-bio.txt,v' | diff -u /dev/null - -L'ossp-pkg/sio/BRAINSTORM/openssl-bio.txt' 2>/dev/null --- ossp-pkg/sio/BRAINSTORM/openssl-bio.txt +++ - 2024-05-15 09:15:08.638218232 +0200 @@ -0,0 +1,1089 @@ + +BIO Routines + +This documentation is rather sparse, you are probably best +off looking at the code for specific details. + +The BIO library is a IO abstraction that was originally +inspired by the need to have callbacks to perform IO to FILE +pointers when using Windows 3.1 DLLs. There are two types +of BIO; a source/sink type and a filter type. +The source/sink methods are as follows: +- BIO_s_mem() memory buffer - a read/write byte array that + grows until memory runs out :-). +- BIO_s_file() FILE pointer - A wrapper around the normal + 'FILE *' commands, good for use with stdin/stdout. +- BIO_s_fd() File descriptor - A wrapper around file + descriptors, often used with pipes. +- BIO_s_socket() Socket - Used around sockets. It is + mostly in the Microsoft world that sockets are different + from file descriptors and there are all those ugly winsock + commands. +- BIO_s_null() Null - read nothing and write nothing.; a + useful endpoint for filter type BIO's specifically things + like the message digest BIO. + +The filter types are +- BIO_f_buffer() IO buffering - does output buffering into + larger chunks and performs input buffering to allow gets() + type functions. +- BIO_f_md() Message digest - a transparent filter that can + be asked to return a message digest for the data that has + passed through it. +- BIO_f_cipher() Encrypt or decrypt all data passing + through the filter. +- BIO_f_base64() Base64 decode on read and encode on write. +- BIO_f_ssl() A filter that performs SSL encryption on the + data sent through it. + +Base BIO functions. +The BIO library has a set of base functions that are +implemented for each particular type. Filter BIOs will +normally call the equivalent function on the source/sink BIO +that they are layered on top of after they have performed +some modification to the data stream. Multiple filter BIOs +can be 'push' into a stack of modifers, so to read from a +file, unbase64 it, then decrypt it, a BIO_f_cipher, +BIO_f_base64 and a BIO_s_file would probably be used. If a +sha-1 and md5 message digest needed to be generated, a stack +two BIO_f_md() BIOs and a BIO_s_null() BIO could be used. +The base functions are +- BIO *BIO_new(BIO_METHOD *type); Create a new BIO of type 'type'. +- int BIO_free(BIO *a); Free a BIO structure. Depending on + the configuration, this will free the underlying data + object for a source/sink BIO. +- int BIO_read(BIO *b, char *data, int len); Read upto 'len' + bytes into 'data'. +- int BIO_gets(BIO *bp,char *buf, int size); Depending on + the BIO, this can either be a 'get special' or a get one + line of data, as per fgets(); +- int BIO_write(BIO *b, char *data, int len); Write 'len' + bytes from 'data' to the 'b' BIO. +- int BIO_puts(BIO *bp,char *buf); Either a 'put special' or + a write null terminated string as per fputs(). +- long BIO_ctrl(BIO *bp,int cmd,long larg,char *parg); A + control function which is used to manipulate the BIO + structure and modify it's state and or report on it. This + function is just about never used directly, rather it + should be used in conjunction with BIO_METHOD specific + macros. +- BIO *BIO_push(BIO *new_top, BIO *old); new_top is apped to the + top of the 'old' BIO list. new_top should be a filter BIO. + All writes will go through 'new_top' first and last on read. + 'old' is returned. +- BIO *BIO_pop(BIO *bio); the new topmost BIO is returned, NULL if + there are no more. + +If a particular low level BIO method is not supported +(normally BIO_gets()), -2 will be returned if that method is +called. Otherwise the IO methods (read, write, gets, puts) +will return the number of bytes read or written, and 0 or -1 +for error (or end of input). For the -1 case, +BIO_should_retry(bio) can be called to determine if it was a +genuine error or a temporary problem. -2 will also be +returned if the BIO has not been initalised yet, in all +cases, the correct error codes are set (accessible via the +ERR library). + + +The following functions are convenience functions: +- int BIO_printf(BIO *bio, char * format, ..); printf but + to a BIO handle. +- long BIO_ctrl_int(BIO *bp,int cmd,long larg,int iarg); a + convenience function to allow a different argument types + to be passed to BIO_ctrl(). +- int BIO_dump(BIO *b,char *bytes,int len); output 'len' + bytes from 'bytes' in a hex dump debug format. +- long BIO_debug_callback(BIO *bio, int cmd, char *argp, int + argi, long argl, long ret) - a default debug BIO callback, + this is mentioned below. To use this one normally has to + use the BIO_set_callback_arg() function to assign an + output BIO for the callback to use. +- BIO *BIO_find_type(BIO *bio,int type); when there is a 'stack' + of BIOs, this function scan the list and returns the first + that is of type 'type', as listed in buffer.h under BIO_TYPE_XXX. +- void BIO_free_all(BIO *bio); Free the bio and all other BIOs + in the list. It walks the bio->next_bio list. + + + +Extra commands are normally implemented as macros calling BIO_ctrl(). +- BIO_number_read(BIO *bio) - the number of bytes processed + by BIO_read(bio,.). +- BIO_number_written(BIO *bio) - the number of bytes written + by BIO_write(bio,.). +- BIO_reset(BIO *bio) - 'reset' the BIO. +- BIO_eof(BIO *bio) - non zero if we are at the current end + of input. +- BIO_set_close(BIO *bio, int close_flag) - set the close flag. +- BIO_get_close(BIO *bio) - return the close flag. + BIO_pending(BIO *bio) - return the number of bytes waiting + to be read (normally buffered internally). +- BIO_flush(BIO *bio) - output any data waiting to be output. +- BIO_should_retry(BIO *io) - after a BIO_read/BIO_write + operation returns 0 or -1, a call to this function will + return non zero if you should retry the call later (this + is for non-blocking IO). +- BIO_should_read(BIO *io) - we should retry when data can + be read. +- BIO_should_write(BIO *io) - we should retry when data can + be written. +- BIO_method_name(BIO *io) - return a string for the method name. +- BIO_method_type(BIO *io) - return the unique ID of the BIO method. +- BIO_set_callback(BIO *io, long (*callback)(BIO *io, int + cmd, char *argp, int argi, long argl, long ret); - sets + the debug callback. +- BIO_get_callback(BIO *io) - return the assigned function + as mentioned above. +- BIO_set_callback_arg(BIO *io, char *arg) - assign some + data against the BIO. This is normally used by the debug + callback but could in reality be used for anything. To + get an idea of how all this works, have a look at the code + in the default debug callback mentioned above. The + callback can modify the return values. + +Details of the BIO_METHOD structure. +typedef struct bio_method_st + { + int type; + char *name; + int (*bwrite)(); + int (*bread)(); + int (*bputs)(); + int (*bgets)(); + long (*ctrl)(); + int (*create)(); + int (*destroy)(); + } BIO_METHOD; + +The 'type' is the numeric type of the BIO, these are listed in buffer.h; +'Name' is a textual representation of the BIO 'type'. +The 7 function pointers point to the respective function +methods, some of which can be NULL if not implemented. +The BIO structure +typedef struct bio_st + { + BIO_METHOD *method; + long (*callback)(BIO * bio, int mode, char *argp, int + argi, long argl, long ret); + char *cb_arg; /* first argument for the callback */ + int init; + int shutdown; + int flags; /* extra storage */ + int num; + char *ptr; + struct bio_st *next_bio; /* used by filter BIOs */ + int references; + unsigned long num_read; + unsigned long num_write; + } BIO; + +- 'Method' is the BIO method. +- 'callback', when configured, is called before and after + each BIO method is called for that particular BIO. This + is intended primarily for debugging and of informational feedback. +- 'init' is 0 when the BIO can be used for operation. + Often, after a BIO is created, a number of operations may + need to be performed before it is available for use. An + example is for BIO_s_sock(). A socket needs to be + assigned to the BIO before it can be used. +- 'shutdown', this flag indicates if the underlying + comunication primative being used should be closed/freed + when the BIO is closed. +- 'flags' is used to hold extra state. It is primarily used + to hold information about why a non-blocking operation + failed and to record startup protocol information for the + SSL BIO. +- 'num' and 'ptr' are used to hold instance specific state + like file descriptors or local data structures. +- 'next_bio' is used by filter BIOs to hold the pointer of the + next BIO in the chain. written data is sent to this BIO and + data read is taken from it. +- 'references' is used to indicate the number of pointers to + this structure. This needs to be '1' before a call to + BIO_free() is made if the BIO_free() function is to + actually free() the structure, otherwise the reference + count is just decreased. The actual BIO subsystem does + not really use this functionality but it is useful when + used in more advanced applicaion. +- num_read and num_write are the total number of bytes + read/written via the 'read()' and 'write()' methods. + +BIO_ctrl operations. +The following is the list of standard commands passed as the +second parameter to BIO_ctrl() and should be supported by +all BIO as best as possible. Some are optional, some are +manditory, in any case, where is makes sense, a filter BIO +should pass such requests to underlying BIO's. +- BIO_CTRL_RESET - Reset the BIO back to an initial state. +- BIO_CTRL_EOF - return 0 if we are not at the end of input, + non 0 if we are. +- BIO_CTRL_INFO - BIO specific special command, normal + information return. +- BIO_CTRL_SET - set IO specific parameter. +- BIO_CTRL_GET - get IO specific parameter. +- BIO_CTRL_GET_CLOSE - Get the close on BIO_free() flag, one + of BIO_CLOSE or BIO_NOCLOSE. +- BIO_CTRL_SET_CLOSE - Set the close on BIO_free() flag. +- BIO_CTRL_PENDING - Return the number of bytes available + for instant reading +- BIO_CTRL_FLUSH - Output pending data, return number of bytes output. +- BIO_CTRL_SHOULD_RETRY - After an IO error (-1 returned) + should we 'retry' when IO is possible on the underlying IO object. +- BIO_CTRL_RETRY_TYPE - What kind of IO are we waiting on. + +The following command is a special BIO_s_file() specific option. +- BIO_CTRL_SET_FILENAME - specify a file to open for IO. + +The BIO_CTRL_RETRY_TYPE needs a little more explanation. +When performing non-blocking IO, or say reading on a memory +BIO, when no data is present (or cannot be written), +BIO_read() and/or BIO_write() will return -1. +BIO_should_retry(bio) will return true if this is due to an +IO condition rather than an actual error. In the case of +BIO_s_mem(), a read when there is no data will return -1 and +a should retry when there is more 'read' data. +The retry type is deduced from 2 macros +BIO_should_read(bio) and BIO_should_write(bio). +Now while it may appear obvious that a BIO_read() failure +should indicate that a retry should be performed when more +read data is available, this is often not true when using +things like an SSL BIO. During the SSL protocol startup +multiple reads and writes are performed, triggered by any +SSL_read or SSL_write. +So to write code that will transparently handle either a +socket or SSL BIO, + i=BIO_read(bio,..) + if (I == -1) + { + if (BIO_should_retry(bio)) + { + if (BIO_should_read(bio)) + { + /* call us again when BIO can be read */ + } + if (BIO_should_write(bio)) + { + /* call us again when BIO can be written */ + } + } + } + +At this point in time only read and write conditions can be +used but in the future I can see the situation for other +conditions, specifically with SSL there could be a condition +of a X509 certificate lookup taking place and so the non- +blocking BIO_read would require a retry when the certificate +lookup subsystem has finished it's lookup. This is all +makes more sense and is easy to use in a event loop type +setup. +When using the SSL BIO, either SSL_read() or SSL_write()s +can be called during the protocol startup and things will +still work correctly. +The nice aspect of the use of the BIO_should_retry() macro +is that all the errno codes that indicate a non-fatal error +are encapsulated in one place. The Windows specific error +codes and WSAGetLastError() calls are also hidden from the +application. + +Notes on each BIO method. +Normally buffer.h is just required but depending on the +BIO_METHOD, ssl.h or evp.h will also be required. + +BIO_METHOD *BIO_s_mem(void); +- BIO_set_mem_buf(BIO *bio, BUF_MEM *bm, int close_flag) - + set the underlying BUF_MEM structure for the BIO to use. +- BIO_get_mem_ptr(BIO *bio, char **pp) - if pp is not NULL, + set it to point to the memory array and return the number + of bytes available. +A read/write BIO. Any data written is appended to the +memory array and any read is read from the front. This BIO +can be used for read/write at the same time. BIO_gets() is +supported in the fgets() sense. +BIO_CTRL_INFO can be used to retrieve pointers to the memory +buffer and it's length. + +BIO_METHOD *BIO_s_file(void); +- BIO_set_fp(BIO *bio, FILE *fp, int close_flag) - set 'FILE *' to use. +- BIO_get_fp(BIO *bio, FILE **fp) - get the 'FILE *' in use. +- BIO_read_filename(BIO *bio, char *name) - read from file. +- BIO_write_filename(BIO *bio, char *name) - write to file. +- BIO_append_filename(BIO *bio, char *name) - append to file. +This BIO sits over the normal system fread()/fgets() type +functions. Gets() is supported. This BIO in theory could be +used for read and write but it is best to think of each BIO +of this type as either a read or a write BIO, not both. + +BIO_METHOD *BIO_s_socket(void); +BIO_METHOD *BIO_s_fd(void); +- BIO_sock_should_retry(int i) - the underlying function + used to determine if a call should be retried; the + argument is the '0' or '-1' returned by the previous BIO + operation. +- BIO_fd_should_retry(int i) - same as the +- BIO_sock_should_retry() except that it is different internally. +- BIO_set_fd(BIO *bio, int fd, int close_flag) - set the + file descriptor to use +- BIO_get_fd(BIO *bio, int *fd) - get the file descriptor. +These two methods are very similar. Gets() is not +supported, if you want this functionality, put a +BIO_f_buffer() onto it. This BIO is bi-directional if the +underlying file descriptor is. This is normally the case +for sockets but not the case for stdio descriptors. + +BIO_METHOD *BIO_s_null(void); +Read and write as much data as you like, it all disappears +into this BIO. + +BIO_METHOD *BIO_f_buffer(void); +- BIO_get_buffer_num_lines(BIO *bio) - return the number of + complete lines in the buffer. +- BIO_set_buffer_size(BIO *bio, long size) - set the size of + the buffers. +This type performs input and output buffering. It performs +both at the same time. The size of the buffer can be set +via the set buffer size option. Data buffered for output is +only written when the buffer fills. + +BIO_METHOD *BIO_f_ssl(void); +- BIO_set_ssl(BIO *bio, SSL *ssl, int close_flag) - the SSL + structure to use. +- BIO_get_ssl(BIO *bio, SSL **ssl) - get the SSL structure + in use. +The SSL bio is a little different from normal BIOs because +the underlying SSL structure is a little different. A SSL +structure performs IO via a read and write BIO. These can +be different and are normally set via the +SSL_set_rbio()/SSL_set_wbio() calls. The SSL_set_fd() calls +are just wrappers that create socket BIOs and then call +SSL_set_bio() where the read and write BIOs are the same. +The BIO_push() operation makes the SSLs IO BIOs the same, so +make sure the BIO pushed is capable of two directional +traffic. If it is not, you will have to install the BIOs +via the more conventional SSL_set_bio() call. BIO_pop() will retrieve +the 'SSL read' BIO. + +BIO_METHOD *BIO_f_md(void); +- BIO_set_md(BIO *bio, EVP_MD *md) - set the message digest + to use. +- BIO_get_md(BIO *bio, EVP_MD **mdp) - return the digest + method in use in mdp, return 0 if not set yet. +- BIO_reset() reinitializes the digest (EVP_DigestInit()) + and passes the reset to the underlying BIOs. +All data read or written via BIO_read() or BIO_write() to +this BIO will be added to the calculated digest. This +implies that this BIO is only one directional. If read and +write operations are performed, two separate BIO_f_md() BIOs +are reuqired to generate digests on both the input and the +output. BIO_gets(BIO *bio, char *md, int size) will place the +generated digest into 'md' and return the number of bytes. +The EVP_MAX_MD_SIZE should probably be used to size the 'md' +array. Reading the digest will also reset it. + +BIO_METHOD *BIO_f_cipher(void); +- BIO_reset() reinitializes the cipher. +- BIO_flush() should be called when the last bytes have been + output to flush the final block of block ciphers. +- BIO_get_cipher_status(BIO *b), when called after the last + read from a cipher BIO, returns non-zero if the data + decrypted correctly, otherwise, 0. +- BIO_set_cipher(BIO *b, EVP_CIPHER *c, unsigned char *key, + unsigned char *iv, int encrypt) This function is used to + setup a cipher BIO. The length of key and iv are + specified by the choice of EVP_CIPHER. Encrypt is 1 to + encrypt and 0 to decrypt. + +BIO_METHOD *BIO_f_base64(void); +- BIO_flush() should be called when the last bytes have been output. +This BIO base64 encodes when writing and base64 decodes when +reading. It will scan the input until a suitable begin line +is found. After reading data, BIO_reset() will reset the +BIO to start scanning again. Do not mix reading and writing +on the same base64 BIO. It is meant as a single stream BIO. + +Directions type +both BIO_s_mem() +one/both BIO_s_file() +both BIO_s_fd() +both BIO_s_socket() +both BIO_s_null() +both BIO_f_buffer() +one BIO_f_md() +one BIO_f_cipher() +one BIO_f_base64() +both BIO_f_ssl() + +It is easy to mix one and two directional BIOs, all one has +to do is to keep two separate BIO pointers for reading and +writing and be careful about usage of underlying BIOs. The +SSL bio by it's very nature has to be two directional but +the BIO_push() command will push the one BIO into the SSL +BIO for both reading and writing. + +The best example program to look at is apps/enc.c and/or perhaps apps/dgst.c. + + +/* crypto/bio/bio.h */ +/* Copyright (C) 1995-1998 Eric Young (eay@cryptsoft.com) + * All rights reserved. + * + * This package is an SSL implementation written + * by Eric Young (eay@cryptsoft.com). + * The implementation was written so as to conform with Netscapes SSL. + * + * This library is free for commercial and non-commercial use as long as + * the following conditions are aheared to. The following conditions + * apply to all code found in this distribution, be it the RC4, RSA, + * lhash, DES, etc., code; not just the SSL code. The SSL documentation + * included with this distribution is covered by the same copyright terms + * except that the holder is Tim Hudson (tjh@cryptsoft.com). + * + * Copyright remains Eric Young's, and as such any Copyright notices in + * the code are not to be removed. + * If this package is used in a product, Eric Young should be given attribution + * as the author of the parts of the library used. + * This can be in the form of a textual message at program startup or + * in documentation (online or textual) provided with the package. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. All advertising materials mentioning features or use of this software + * must display the following acknowledgement: + * "This product includes cryptographic software written by + * Eric Young (eay@cryptsoft.com)" + * The word 'cryptographic' can be left out if the rouines from the library + * being used are not cryptographic related :-). + * 4. If you include any Windows specific code (or a derivative thereof) from + * the apps directory (application code) you must include an acknowledgement: + * "This product includes software written by Tim Hudson (tjh@cryptsoft.com)" + * + * THIS SOFTWARE IS PROVIDED BY ERIC YOUNG ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * The licence and distribution terms for any publically available version or + * derivative of this code cannot be changed. i.e. this code cannot simply be + * copied and put under another distribution licence + * [including the GNU Public Licence.] + */ + +#ifndef HEADER_BIO_H +#define HEADER_BIO_H + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include +#include + +/* These are the 'types' of BIOs */ +#define BIO_TYPE_NONE 0 +#define BIO_TYPE_MEM (1|0x0400) +#define BIO_TYPE_FILE (2|0x0400) + +#define BIO_TYPE_FD (4|0x0400|0x0100) +#define BIO_TYPE_SOCKET (5|0x0400|0x0100) +#define BIO_TYPE_NULL (6|0x0400) +#define BIO_TYPE_SSL (7|0x0200) +#define BIO_TYPE_MD (8|0x0200) /* pasive filter */ +#define BIO_TYPE_BUFFER (9|0x0200) /* filter */ +#define BIO_TYPE_CIPHER (10|0x0200) /* filter */ +#define BIO_TYPE_BASE64 (11|0x0200) /* filter */ +#define BIO_TYPE_CONNECT (12|0x0400|0x0100) /* socket - connect */ +#define BIO_TYPE_ACCEPT (13|0x0400|0x0100) /* socket for accept */ +#define BIO_TYPE_PROXY_CLIENT (14|0x0200) /* client proxy BIO */ +#define BIO_TYPE_PROXY_SERVER (15|0x0200) /* server proxy BIO */ +#define BIO_TYPE_NBIO_TEST (16|0x0200) /* server proxy BIO */ +#define BIO_TYPE_NULL_FILTER (17|0x0200) +#define BIO_TYPE_BER (18|0x0200) /* BER -> bin filter */ +#define BIO_TYPE_BIO (19|0x0400) /* (half a) BIO pair */ + +#define BIO_TYPE_DESCRIPTOR 0x0100 /* socket, fd, connect or accept */ +#define BIO_TYPE_FILTER 0x0200 +#define BIO_TYPE_SOURCE_SINK 0x0400 + +/* BIO_FILENAME_READ|BIO_CLOSE to open or close on free. + * BIO_set_fp(in,stdin,BIO_NOCLOSE); */ +#define BIO_NOCLOSE 0x00 +#define BIO_CLOSE 0x01 + +/* These are used in the following macros and are passed to + * BIO_ctrl() */ +#define BIO_CTRL_RESET 1 /* opt - rewind/zero etc */ +#define BIO_CTRL_EOF 2 /* opt - are we at the eof */ +#define BIO_CTRL_INFO 3 /* opt - extra tit-bits */ +#define BIO_CTRL_SET 4 /* man - set the 'IO' type */ +#define BIO_CTRL_GET 5 /* man - get the 'IO' type */ +#define BIO_CTRL_PUSH 6 /* opt - internal, used to signify change */ +#define BIO_CTRL_POP 7 /* opt - internal, used to signify change */ +#define BIO_CTRL_GET_CLOSE 8 /* man - set the 'close' on free */ +#define BIO_CTRL_SET_CLOSE 9 /* man - set the 'close' on free */ +#define BIO_CTRL_PENDING 10 /* opt - is their more data buffered */ +#define BIO_CTRL_FLUSH 11 /* opt - 'flush' buffered output */ +#define BIO_CTRL_DUP 12 /* man - extra stuff for 'duped' BIO */ +#define BIO_CTRL_WPENDING 13 /* opt - number of bytes still to write */ +/* callback is int cb(BIO *bio,state,ret); */ +#define BIO_CTRL_SET_CALLBACK 14 /* opt - set callback function */ +#define BIO_CTRL_GET_CALLBACK 15 /* opt - set callback function */ + +#define BIO_CTRL_SET_FILENAME 30 /* BIO_s_file special */ + +/* modifiers */ +#define BIO_FP_READ 0x02 +#define BIO_FP_WRITE 0x04 +#define BIO_FP_APPEND 0x08 +#define BIO_FP_TEXT 0x10 + +#define BIO_FLAGS_READ 0x01 +#define BIO_FLAGS_WRITE 0x02 +#define BIO_FLAGS_IO_SPECIAL 0x04 +#define BIO_FLAGS_RWS (BIO_FLAGS_READ|BIO_FLAGS_WRITE|BIO_FLAGS_IO_SPECIAL) +#define BIO_FLAGS_SHOULD_RETRY 0x08 + +/* Used in BIO_gethostbyname() */ +#define BIO_GHBN_CTRL_HITS 1 +#define BIO_GHBN_CTRL_MISSES 2 +#define BIO_GHBN_CTRL_CACHE_SIZE 3 +#define BIO_GHBN_CTRL_GET_ENTRY 4 +#define BIO_GHBN_CTRL_FLUSH 5 + +/* Mostly used in the SSL BIO */ +/* Not used anymore + * #define BIO_FLAGS_PROTOCOL_DELAYED_READ 0x10 + * #define BIO_FLAGS_PROTOCOL_DELAYED_WRITE 0x20 + * #define BIO_FLAGS_PROTOCOL_STARTUP 0x40 + */ + +#define BIO_FLAGS_BASE64_NO_NL 0x100 + +/* This is used with memory BIOs: it means we shouldn't free up or change the + * data in any way. + */ +#define BIO_FLAGS_MEM_RDONLY 0x200 + +#define BIO_set_flags(b,f) ((b)->flags|=(f)) +#define BIO_get_flags(b) ((b)->flags) +#define BIO_set_retry_special(b) \ + ((b)->flags|=(BIO_FLAGS_IO_SPECIAL|BIO_FLAGS_SHOULD_RETRY)) +#define BIO_set_retry_read(b) \ + ((b)->flags|=(BIO_FLAGS_READ|BIO_FLAGS_SHOULD_RETRY)) +#define BIO_set_retry_write(b) \ + ((b)->flags|=(BIO_FLAGS_WRITE|BIO_FLAGS_SHOULD_RETRY)) + +/* These are normally used internally in BIOs */ +#define BIO_clear_flags(b,f) ((b)->flags&= ~(f)) +#define BIO_clear_retry_flags(b) \ + ((b)->flags&= ~(BIO_FLAGS_RWS|BIO_FLAGS_SHOULD_RETRY)) +#define BIO_get_retry_flags(b) \ + ((b)->flags&(BIO_FLAGS_RWS|BIO_FLAGS_SHOULD_RETRY)) + +/* These shouldbe used by the application to tell why we should retry */ +#define BIO_should_read(a) ((a)->flags & BIO_FLAGS_READ) +#define BIO_should_write(a) ((a)->flags & BIO_FLAGS_WRITE) +#define BIO_should_io_special(a) ((a)->flags & BIO_FLAGS_IO_SPECIAL) +#define BIO_retry_type(a) ((a)->flags & BIO_FLAGS_RWS) +#define BIO_should_retry(a) ((a)->flags & BIO_FLAGS_SHOULD_RETRY) + +/* The next two are used in conjunction with the + * BIO_should_io_special() condition. After this returns true, + * BIO *BIO_get_retry_BIO(BIO *bio, int *reason); will walk the BIO + * stack and return the 'reason' for the special and the offending BIO. + * Given a BIO, BIO_get_retry_reason(bio) will return the code. */ +/* Returned from the SSL bio when the certificate retrieval code had an error */ +#define BIO_RR_SSL_X509_LOOKUP 0x01 +/* Returned from the connect BIO when a connect would have blocked */ +#define BIO_RR_CONNECT 0x02 + +/* These are passed by the BIO callback */ +#define BIO_CB_FREE 0x01 +#define BIO_CB_READ 0x02 +#define BIO_CB_WRITE 0x03 +#define BIO_CB_PUTS 0x04 +#define BIO_CB_GETS 0x05 +#define BIO_CB_CTRL 0x06 + +/* The callback is called before and after the underling operation, + * The BIO_CB_RETURN flag indicates if it is after the call */ +#define BIO_CB_RETURN 0x80 +#define BIO_CB_return(a) ((a)|BIO_CB_RETURN)) +#define BIO_cb_pre(a) (!((a)&BIO_CB_RETURN)) +#define BIO_cb_post(a) ((a)&BIO_CB_RETURN) + +#define BIO_set_callback(b,cb) ((b)->callback=(cb)) +#define BIO_set_callback_arg(b,arg) ((b)->cb_arg=(char *)(arg)) +#define BIO_get_callback_arg(b) ((b)->cb_arg) +#define BIO_get_callback(b) ((b)->callback) +#define BIO_method_name(b) ((b)->method->name) +#define BIO_method_type(b) ((b)->method->type) + +#ifndef WIN16 +typedef struct bio_method_st + { + int type; + const char *name; + int (*bwrite)(); + int (*bread)(); + int (*bputs)(); + int (*bgets)(); + long (*ctrl)(); + int (*create)(); + int (*destroy)(); + } BIO_METHOD; +#else +typedef struct bio_method_st + { + int type; + const char *name; + int (_far *bwrite)(); + int (_far *bread)(); + int (_far *bputs)(); + int (_far *bgets)(); + long (_far *ctrl)(); + int (_far *create)(); + int (_far *destroy)(); + } BIO_METHOD; +#endif + +typedef struct bio_st + { + BIO_METHOD *method; + /* bio, mode, argp, argi, argl, ret */ + long (*callback)(struct bio_st *,int,const char *,int, long,long); + char *cb_arg; /* first argument for the callback */ + + int init; + int shutdown; + int flags; /* extra storage */ + int retry_reason; + int num; + void *ptr; + struct bio_st *next_bio; /* used by filter BIOs */ + struct bio_st *prev_bio; /* used by filter BIOs */ + int references; + unsigned long num_read; + unsigned long num_write; + + CRYPTO_EX_DATA ex_data; + } BIO; + +typedef struct bio_f_buffer_ctx_struct + { + /* BIO *bio; */ /* this is now in the BIO struct */ + int ibuf_size; /* how big is the input buffer */ + int obuf_size; /* how big is the output buffer */ + + char *ibuf; /* the char array */ + int ibuf_len; /* how many bytes are in it */ + int ibuf_off; /* write/read offset */ + + char *obuf; /* the char array */ + int obuf_len; /* how many bytes are in it */ + int obuf_off; /* write/read offset */ + } BIO_F_BUFFER_CTX; + +/* connect BIO stuff */ +#define BIO_CONN_S_BEFORE 1 +#define BIO_CONN_S_GET_IP 2 +#define BIO_CONN_S_GET_PORT 3 +#define BIO_CONN_S_CREATE_SOCKET 4 +#define BIO_CONN_S_CONNECT 5 +#define BIO_CONN_S_OK 6 +#define BIO_CONN_S_BLOCKED_CONNECT 7 +#define BIO_CONN_S_NBIO 8 +/*#define BIO_CONN_get_param_hostname BIO_ctrl */ + +#define BIO_number_read(b) ((b)->num_read) +#define BIO_number_written(b) ((b)->num_write) + +#define BIO_C_SET_CONNECT 100 +#define BIO_C_DO_STATE_MACHINE 101 +#define BIO_C_SET_NBIO 102 +#define BIO_C_SET_PROXY_PARAM 103 +#define BIO_C_SET_FD 104 +#define BIO_C_GET_FD 105 +#define BIO_C_SET_FILE_PTR 106 +#define BIO_C_GET_FILE_PTR 107 +#define BIO_C_SET_FILENAME 108 +#define BIO_C_SET_SSL 109 +#define BIO_C_GET_SSL 110 +#define BIO_C_SET_MD 111 +#define BIO_C_GET_MD 112 +#define BIO_C_GET_CIPHER_STATUS 113 +#define BIO_C_SET_BUF_MEM 114 +#define BIO_C_GET_BUF_MEM_PTR 115 +#define BIO_C_GET_BUFF_NUM_LINES 116 +#define BIO_C_SET_BUFF_SIZE 117 +#define BIO_C_SET_ACCEPT 118 +#define BIO_C_SSL_MODE 119 +#define BIO_C_GET_MD_CTX 120 +#define BIO_C_GET_PROXY_PARAM 121 +#define BIO_C_SET_BUFF_READ_DATA 122 /* data to read first */ +#define BIO_C_GET_CONNECT 123 +#define BIO_C_GET_ACCEPT 124 +#define BIO_C_SET_SSL_RENEGOTIATE_BYTES 125 +#define BIO_C_GET_SSL_NUM_RENEGOTIATES 126 +#define BIO_C_SET_SSL_RENEGOTIATE_TIMEOUT 127 +#define BIO_C_FILE_SEEK 128 +#define BIO_C_GET_CIPHER_CTX 129 +#define BIO_C_SET_BUF_MEM_EOF_RETURN 130/*return end of input value*/ +#define BIO_C_SET_BIND_MODE 131 +#define BIO_C_GET_BIND_MODE 132 +#define BIO_C_FILE_TELL 133 +#define BIO_C_GET_SOCKS 134 +#define BIO_C_SET_SOCKS 135 + +#define BIO_C_SET_WRITE_BUF_SIZE 136/* for BIO_s_bio */ +#define BIO_C_GET_WRITE_BUF_SIZE 137 +#define BIO_C_MAKE_BIO_PAIR 138 +#define BIO_C_DESTROY_BIO_PAIR 139 +#define BIO_C_GET_WRITE_GUARANTEE 140 +#define BIO_C_GET_READ_REQUEST 141 +#define BIO_C_SHUTDOWN_WR 142 +#define BIO_C_NREAD0 143 +#define BIO_C_NREAD 144 +#define BIO_C_NWRITE0 145 +#define BIO_C_NWRITE 146 +#define BIO_C_RESET_READ_REQUEST 147 + + +#define BIO_set_app_data(s,arg) BIO_set_ex_data(s,0,(char *)arg) +#define BIO_get_app_data(s) BIO_get_ex_data(s,0) + +/* BIO_s_connect() and BIO_s_socks4a_connect() */ +#define BIO_set_conn_hostname(b,name) BIO_ctrl(b,BIO_C_SET_CONNECT,0,(char *)name) +#define BIO_set_conn_port(b,port) BIO_ctrl(b,BIO_C_SET_CONNECT,1,(char *)port) +#define BIO_set_conn_ip(b,ip) BIO_ctrl(b,BIO_C_SET_CONNECT,2,(char *)ip) +#define BIO_set_conn_int_port(b,port) BIO_ctrl(b,BIO_C_SET_CONNECT,3,(char *)port) +#define BIO_get_conn_hostname(b) BIO_ptr_ctrl(b,BIO_C_GET_CONNECT,0) +#define BIO_get_conn_port(b) BIO_ptr_ctrl(b,BIO_C_GET_CONNECT,1) +#define BIO_get_conn_ip(b,ip) BIO_ptr_ctrl(b,BIO_C_SET_CONNECT,2) +#define BIO_get_conn_int_port(b,port) BIO_int_ctrl(b,BIO_C_SET_CONNECT,3,port) + + +#define BIO_set_nbio(b,n) BIO_ctrl(b,BIO_C_SET_NBIO,(n),NULL) + +/* BIO_s_accept_socket() */ +#define BIO_set_accept_port(b,name) BIO_ctrl(b,BIO_C_SET_ACCEPT,0,(char *)name) +#define BIO_get_accept_port(b) BIO_ptr_ctrl(b,BIO_C_GET_ACCEPT,0) +/* #define BIO_set_nbio(b,n) BIO_ctrl(b,BIO_C_SET_NBIO,(n),NULL) */ +#define BIO_set_nbio_accept(b,n) BIO_ctrl(b,BIO_C_SET_ACCEPT,1,(n)?"a":NULL) +#define BIO_set_accept_bios(b,bio) BIO_ctrl(b,BIO_C_SET_ACCEPT,2,(char *)bio) + +#define BIO_BIND_NORMAL 0 +#define BIO_BIND_REUSEADDR_IF_UNUSED 1 +#define BIO_BIND_REUSEADDR 2 +#define BIO_set_bind_mode(b,mode) BIO_ctrl(b,BIO_C_SET_BIND_MODE,mode,NULL) +#define BIO_get_bind_mode(b,mode) BIO_ctrl(b,BIO_C_GET_BIND_MODE,0,NULL) + +#define BIO_do_connect(b) BIO_do_handshake(b) +#define BIO_do_accept(b) BIO_do_handshake(b) +#define BIO_do_handshake(b) BIO_ctrl(b,BIO_C_DO_STATE_MACHINE,0,NULL) + +/* BIO_s_proxy_client() */ +#define BIO_set_url(b,url) BIO_ctrl(b,BIO_C_SET_PROXY_PARAM,0,(char *)(url)) +#define BIO_set_proxies(b,p) BIO_ctrl(b,BIO_C_SET_PROXY_PARAM,1,(char *)(p)) +/* BIO_set_nbio(b,n) */ +#define BIO_set_filter_bio(b,s) BIO_ctrl(b,BIO_C_SET_PROXY_PARAM,2,(char *)(s)) +/* BIO *BIO_get_filter_bio(BIO *bio); */ +#define BIO_set_proxy_cb(b,cb) BIO_ctrl(b,BIO_C_SET_PROXY_PARAM,3,(char *)(cb)) +#define BIO_set_proxy_header(b,sk) BIO_ctrl(b,BIO_C_SET_PROXY_PARAM,4,(char *)sk) +#define BIO_set_no_connect_return(b,bool) BIO_int_ctrl(b,BIO_C_SET_PROXY_PARAM,5,bool) + +#define BIO_get_proxy_header(b,skp) BIO_ctrl(b,BIO_C_GET_PROXY_PARAM,0,(char *)skp) +#define BIO_get_proxies(b,pxy_p) BIO_ctrl(b,BIO_C_GET_PROXY_PARAM,1,(char *)(pxy_p)) +#define BIO_get_url(b,url) BIO_ctrl(b,BIO_C_GET_PROXY_PARAM,2,(char *)(url)) +#define BIO_get_no_connect_return(b) BIO_ctrl(b,BIO_C_GET_PROXY_PARAM,5,NULL) + +#define BIO_set_fd(b,fd,c) BIO_int_ctrl(b,BIO_C_SET_FD,c,fd) +#define BIO_get_fd(b,c) BIO_ctrl(b,BIO_C_GET_FD,0,(char *)c) + +#define BIO_set_fp(b,fp,c) BIO_ctrl(b,BIO_C_SET_FILE_PTR,c,(char *)fp) +#define BIO_get_fp(b,fpp) BIO_ctrl(b,BIO_C_GET_FILE_PTR,0,(char *)fpp) + +#define BIO_seek(b,ofs) (int)BIO_ctrl(b,BIO_C_FILE_SEEK,ofs,NULL) +#define BIO_tell(b) (int)BIO_ctrl(b,BIO_C_FILE_TELL,0,NULL) + +/* name is cast to lose const, but might be better to route through a function + so we can do it safely */ +#ifdef CONST_STRICT +/* If you are wondering why this isn't defined, its because CONST_STRICT is + * purely a compile-time kludge to allow const to be checked. + */ +int BIO_read_filename(BIO *b,const char *name); +#else +#define BIO_read_filename(b,name) BIO_ctrl(b,BIO_C_SET_FILENAME, \ + BIO_CLOSE|BIO_FP_READ,(char *)name) +#endif +#define BIO_write_filename(b,name) BIO_ctrl(b,BIO_C_SET_FILENAME, \ + BIO_CLOSE|BIO_FP_WRITE,name) +#define BIO_append_filename(b,name) BIO_ctrl(b,BIO_C_SET_FILENAME, \ + BIO_CLOSE|BIO_FP_APPEND,name) +#define BIO_rw_filename(b,name) BIO_ctrl(b,BIO_C_SET_FILENAME, \ + BIO_CLOSE|BIO_FP_READ|BIO_FP_WRITE,name) + +/* WARNING WARNING, this ups the reference count on the read bio of the + * SSL structure. This is because the ssl read BIO is now pointed to by + * the next_bio field in the bio. So when you free the BIO, make sure + * you are doing a BIO_free_all() to catch the underlying BIO. */ +#define BIO_set_ssl(b,ssl,c) BIO_ctrl(b,BIO_C_SET_SSL,c,(char *)ssl) +#define BIO_get_ssl(b,sslp) BIO_ctrl(b,BIO_C_GET_SSL,0,(char *)sslp) +#define BIO_set_ssl_mode(b,client) BIO_ctrl(b,BIO_C_SSL_MODE,client,NULL) +#define BIO_set_ssl_renegotiate_bytes(b,num) \ + BIO_ctrl(b,BIO_C_SET_SSL_RENEGOTIATE_BYTES,num,NULL); +#define BIO_get_num_renegotiates(b) \ + BIO_ctrl(b,BIO_C_SET_SSL_NUM_RENEGOTIATES,0,NULL); +#define BIO_set_ssl_renegotiate_timeout(b,seconds) \ + BIO_ctrl(b,BIO_C_SET_SSL_RENEGOTIATE_TIMEOUT,seconds,NULL); + +/* defined in evp.h */ +/* #define BIO_set_md(b,md) BIO_ctrl(b,BIO_C_SET_MD,1,(char *)md) */ + +#define BIO_get_mem_data(b,pp) BIO_ctrl(b,BIO_CTRL_INFO,0,(char *)pp) +#define BIO_set_mem_buf(b,bm,c) BIO_ctrl(b,BIO_C_SET_BUF_MEM,c,(char *)bm) +#define BIO_get_mem_ptr(b,pp) BIO_ctrl(b,BIO_C_GET_BUF_MEM_PTR,0,(char *)pp) +#define BIO_set_mem_eof_return(b,v) \ + BIO_ctrl(b,BIO_C_SET_BUF_MEM_EOF_RETURN,v,NULL) + +/* For the BIO_f_buffer() type */ +#define BIO_get_buffer_num_lines(b) BIO_ctrl(b,BIO_C_GET_BUFF_NUM_LINES,0,NULL) +#define BIO_set_buffer_size(b,size) BIO_ctrl(b,BIO_C_SET_BUFF_SIZE,size,NULL) +#define BIO_set_read_buffer_size(b,size) BIO_int_ctrl(b,BIO_C_SET_BUFF_SIZE,size,0) +#define BIO_set_write_buffer_size(b,size) BIO_int_ctrl(b,BIO_C_SET_BUFF_SIZE,size,1) +#define BIO_set_buffer_read_data(b,buf,num) BIO_ctrl(b,BIO_C_SET_BUFF_READ_DATA,num,buf) + +/* Don't use the next one unless you know what you are doing :-) */ +#define BIO_dup_state(b,ret) BIO_ctrl(b,BIO_CTRL_DUP,0,(char *)(ret)) + +#define BIO_reset(b) (int)BIO_ctrl(b,BIO_CTRL_RESET,0,NULL) +#define BIO_eof(b) (int)BIO_ctrl(b,BIO_CTRL_EOF,0,NULL) +#define BIO_set_close(b,c) (int)BIO_ctrl(b,BIO_CTRL_SET_CLOSE,(c),NULL) +#define BIO_get_close(b) (int)BIO_ctrl(b,BIO_CTRL_GET_CLOSE,0,NULL) +#define BIO_pending(b) (int)BIO_ctrl(b,BIO_CTRL_PENDING,0,NULL) +#define BIO_wpending(b) (int)BIO_ctrl(b,BIO_CTRL_WPENDING,0,NULL) +/* ...pending macros have inappropriate return type */ +size_t BIO_ctrl_pending(BIO *b); +size_t BIO_ctrl_wpending(BIO *b); +#define BIO_flush(b) (int)BIO_ctrl(b,BIO_CTRL_FLUSH,0,NULL) +#define BIO_get_info_callback(b,cbp) (int)BIO_ctrl(b,BIO_CTRL_GET_CALLBACK,0,(char *)cbp) +#define BIO_set_info_callback(b,cb) (int)BIO_ctrl(b,BIO_CTRL_SET_CALLBACK,0,(char *)cb) + +/* For the BIO_f_buffer() type */ +#define BIO_buffer_get_num_lines(b) BIO_ctrl(b,BIO_CTRL_GET,0,NULL) + +/* For BIO_s_bio() */ +#define BIO_set_write_buf_size(b,size) (int)BIO_ctrl(b,BIO_C_SET_WRITE_BUF_SIZE,size,NULL) +#define BIO_get_write_buf_size(b,size) (size_t)BIO_ctrl(b,BIO_C_GET_WRITE_BUF_SIZE,size,NULL) +#define BIO_make_bio_pair(b1,b2) (int)BIO_ctrl(b1,BIO_C_MAKE_BIO_PAIR,0,b2) +#define BIO_destroy_bio_pair(b) (int)BIO_ctrl(b,BIO_C_DESTROY_BIO_PAIR,0,NULL) +/* macros with inappropriate type -- but ...pending macros use int too: */ +#define BIO_get_write_guarantee(b) (int)BIO_ctrl(b,BIO_C_GET_WRITE_GUARANTEE,0,NULL) +#define BIO_get_read_request(b) (int)BIO_ctrl(b,BIO_C_GET_READ_REQUEST,0,NULL) +size_t BIO_ctrl_get_write_guarantee(BIO *b); +size_t BIO_ctrl_get_read_request(BIO *b); +int BIO_ctrl_reset_read_request(BIO *b); + +#ifdef NO_STDIO +#define NO_FP_API +#endif + + +/* These two aren't currently implemented */ +/* int BIO_get_ex_num(BIO *bio); */ +/* void BIO_set_ex_free_func(BIO *bio,int idx,void (*cb)()); */ +int BIO_set_ex_data(BIO *bio,int idx,char *data); +char *BIO_get_ex_data(BIO *bio,int idx); +int BIO_get_ex_new_index(long argl, char *argp, int (*new_func)(), + int (*dup_func)(), void (*free_func)()); + +# if defined(WIN16) && defined(_WINDLL) +BIO_METHOD *BIO_s_file_internal(void); +BIO *BIO_new_file_internal(char *filename, char *mode); +BIO *BIO_new_fp_internal(FILE *stream, int close_flag); +# define BIO_s_file BIO_s_file_internal +# define BIO_new_file BIO_new_file_internal +# define BIO_new_fp BIO_new_fp_internal +# else /* FP_API */ +BIO_METHOD *BIO_s_file(void ); +BIO *BIO_new_file(const char *filename, const char *mode); +BIO *BIO_new_fp(FILE *stream, int close_flag); +# define BIO_s_file_internal BIO_s_file +# define BIO_new_file_internal BIO_new_file +# define BIO_new_fp_internal BIO_s_file +# endif /* FP_API */ +BIO * BIO_new(BIO_METHOD *type); +int BIO_set(BIO *a,BIO_METHOD *type); +int BIO_free(BIO *a); +int BIO_read(BIO *b, void *data, int len); +int BIO_gets(BIO *bp,char *buf, int size); +int BIO_write(BIO *b, const char *data, int len); +int BIO_puts(BIO *bp,const char *buf); +long BIO_ctrl(BIO *bp,int cmd,long larg,void *parg); +char * BIO_ptr_ctrl(BIO *bp,int cmd,long larg); +long BIO_int_ctrl(BIO *bp,int cmd,long larg,int iarg); +BIO * BIO_push(BIO *b,BIO *append); +BIO * BIO_pop(BIO *b); +void BIO_free_all(BIO *a); +BIO * BIO_find_type(BIO *b,int bio_type); +BIO * BIO_get_retry_BIO(BIO *bio, int *reason); +int BIO_get_retry_reason(BIO *bio); +BIO * BIO_dup_chain(BIO *in); + +int BIO_nread0(BIO *bio, char **buf); +int BIO_nread(BIO *bio, char **buf, int num); +int BIO_nwrite0(BIO *bio, char **buf); +int BIO_nwrite(BIO *bio, char **buf, int num); + +#ifndef WIN16 +long BIO_debug_callback(BIO *bio,int cmd,const char *argp,int argi, + long argl,long ret); +#else +long _far _loadds BIO_debug_callback(BIO *bio,int cmd,const char *argp,int argi, + long argl,long ret); +#endif + +BIO_METHOD *BIO_s_mem(void); +BIO *BIO_new_mem_buf(void *buf, int len); +BIO_METHOD *BIO_s_socket(void); +BIO_METHOD *BIO_s_connect(void); +BIO_METHOD *BIO_s_accept(void); +BIO_METHOD *BIO_s_fd(void); +BIO_METHOD *BIO_s_log(void); +BIO_METHOD *BIO_s_bio(void); +BIO_METHOD *BIO_s_null(void); +BIO_METHOD *BIO_f_null(void); +BIO_METHOD *BIO_f_buffer(void); +BIO_METHOD *BIO_f_nbio_test(void); +/* BIO_METHOD *BIO_f_ber(void); */ + +int BIO_sock_should_retry(int i); +int BIO_sock_non_fatal_error(int error); +int BIO_fd_should_retry(int i); +int BIO_fd_non_fatal_error(int error); +int BIO_dump(BIO *b,const char *bytes,int len); + +struct hostent *BIO_gethostbyname(const char *name); +/* We might want a thread-safe interface too: + * struct hostent *BIO_gethostbyname_r(const char *name, + * struct hostent *result, void *buffer, size_t buflen); + * or something similar (caller allocates a struct hostent, + * pointed to by "result", and additional buffer space for the various + * substructures; if the buffer does not suffice, NULL is returned + * and an appropriate error code is set). + */ +int BIO_sock_error(int sock); +int BIO_socket_ioctl(int fd, long type, unsigned long *arg); +int BIO_socket_nbio(int fd,int mode); +int BIO_get_port(const char *str, unsigned short *port_ptr); +int BIO_get_host_ip(const char *str, unsigned char *ip); +int BIO_get_accept_socket(char *host_port,int mode); +int BIO_accept(int sock,char **ip_port); +int BIO_sock_init(void ); +void BIO_sock_cleanup(void); +int BIO_set_tcp_ndelay(int sock,int turn_on); + +void ERR_load_BIO_strings(void ); + +BIO *BIO_new_socket(int sock, int close_flag); +BIO *BIO_new_fd(int fd, int close_flag); +BIO *BIO_new_connect(char *host_port); +BIO *BIO_new_accept(char *host_port); + +int BIO_new_bio_pair(BIO **bio1, size_t writebuf1, + BIO **bio2, size_t writebuf2); +/* If successful, returns 1 and in *bio1, *bio2 two BIO pair endpoints. + * Otherwise returns 0 and sets *bio1 and *bio2 to NULL. + * Size 0 uses default value. + */ + +void BIO_copy_next_retry(BIO *b); + +long BIO_ghbn_ctrl(int cmd,int iarg,char *parg); + +int BIO_printf(BIO *bio, ...); + +/* BEGIN ERROR CODES */ +/* The following lines are auto generated by the script mkerr.pl. Any changes + * made after this point may be overwritten when the script is next run. + */ + +/* Error codes for the BIO functions. */ + +/* Function codes. */ +#define BIO_F_ACPT_STATE 100 +#define BIO_F_BIO_ACCEPT 101 +#define BIO_F_BIO_BER_GET_HEADER 102 +#define BIO_F_BIO_CTRL 103 +#define BIO_F_BIO_GETHOSTBYNAME 120 +#define BIO_F_BIO_GETS 104 +#define BIO_F_BIO_GET_ACCEPT_SOCKET 105 +#define BIO_F_BIO_GET_HOST_IP 106 +#define BIO_F_BIO_GET_PORT 107 +#define BIO_F_BIO_MAKE_PAIR 121 +#define BIO_F_BIO_NEW 108 +#define BIO_F_BIO_NEW_FILE 109 +#define BIO_F_BIO_NEW_MEM_BUF 126 +#define BIO_F_BIO_NREAD 123 +#define BIO_F_BIO_NREAD0 124 +#define BIO_F_BIO_NWRITE 125 +#define BIO_F_BIO_NWRITE0 122 +#define BIO_F_BIO_PUTS 110 +#define BIO_F_BIO_READ 111 +#define BIO_F_BIO_SOCK_INIT 112 +#define BIO_F_BIO_WRITE 113 +#define BIO_F_BUFFER_CTRL 114 +#define BIO_F_CONN_STATE 115 +#define BIO_F_FILE_CTRL 116 +#define BIO_F_MEM_WRITE 117 +#define BIO_F_SSL_NEW 118 +#define BIO_F_WSASTARTUP 119 + +/* Reason codes. */ +#define BIO_R_ACCEPT_ERROR 100 +#define BIO_R_BAD_FOPEN_MODE 101 +#define BIO_R_BAD_HOSTNAME_LOOKUP 102 +#define BIO_R_BROKEN_PIPE 124 +#define BIO_R_CONNECT_ERROR 103 +#define BIO_R_ERROR_SETTING_NBIO 104 +#define BIO_R_ERROR_SETTING_NBIO_ON_ACCEPTED_SOCKET 105 +#define BIO_R_ERROR_SETTING_NBIO_ON_ACCEPT_SOCKET 106 +#define BIO_R_GETHOSTBYNAME_ADDR_IS_NOT_AF_INET 107 +#define BIO_R_INVALID_ARGUMENT 125 +#define BIO_R_INVALID_IP_ADDRESS 108 +#define BIO_R_IN_USE 123 +#define BIO_R_KEEPALIVE 109 +#define BIO_R_NBIO_CONNECT_ERROR 110 +#define BIO_R_NO_ACCEPT_PORT_SPECIFIED 111 +#define BIO_R_NO_HOSTNAME_SPECIFIED 112 +#define BIO_R_NO_PORT_DEFINED 113 +#define BIO_R_NO_PORT_SPECIFIED 114 +#define BIO_R_NULL_PARAMETER 115 +#define BIO_R_TAG_MISMATCH 116 +#define BIO_R_UNABLE_TO_BIND_SOCKET 117 +#define BIO_R_UNABLE_TO_CREATE_SOCKET 118 +#define BIO_R_UNABLE_TO_LISTEN_SOCKET 119 +#define BIO_R_UNINITIALIZED 120 +#define BIO_R_UNSUPPORTED_METHOD 121 +#define BIO_R_WRITE_TO_READ_ONLY_BIO 126 +#define BIO_R_WSASTARTUP 122 + +#ifdef __cplusplus +} +#endif +#endif + Index: ossp-pkg/sio/BRAINSTORM/panos-sio.txt RCS File: /v/ossp/cvs/ossp-pkg/sio/BRAINSTORM/panos-sio.txt,v co -q -kk -p'1.1' '/v/ossp/cvs/ossp-pkg/sio/BRAINSTORM/panos-sio.txt,v' | diff -u /dev/null - -L'ossp-pkg/sio/BRAINSTORM/panos-sio.txt' 2>/dev/null --- ossp-pkg/sio/BRAINSTORM/panos-sio.txt +++ - 2024-05-15 09:15:08.641431509 +0200 @@ -0,0 +1,462 @@ + + + +SIO(3X) SIO(3X) + + +NNAAMMEE + Sread, Sgetc, Srdline, Sfetch, Swrite, Sputc, Sprint, + Sprintv, Sdone, Sundo, Stie, Suntie, Sflush, Sclose, + Sbuftype, Smorefds, Sgetchar, Sputchar, SIOLINELEN, + SIOMAXLINELEN - fast stream I/O + +SSYYNNOOPPSSIISS + ##iinncclluuddee ""ssiioo..hh"" + + iinntt SSrreeaadd(( ffdd,, bbuuff,, nnbbyytteess )) + iinntt ffdd ;; + cchhaarr **bbuuff ;; + iinntt nnbbyytteess ;; + + iinntt SSggeettcc(( ffdd )) + iinntt ffdd ;; + + cchhaarr **SSrrddlliinnee(( ffdd )) + iinntt ffdd ;; + + cchhaarr **SSffeettcchh(( ffdd,, lleennggtthh )) + iinntt ffdd ;; + lloonngg **lleennggtthh ;; + + iinntt SSwwrriittee(( ffdd,, bbuuff,, nnbbyytteess )) + iinntt ffdd ;; + cchhaarr **bbuuff ;; + iinntt nnbbyytteess ;; + + iinntt SSppuuttcc(( ffdd,, cc )) + iinntt ffdd ;; + cchhaarr cc ;; + + iinntt SSpprriinntt(( ffdd,, ffoorrmmaatt [[ ,, ...... ]] )) + iinntt ffdd ;; + cchhaarr **ffoorrmmaatt ;; + + iinntt SSpprriinnttvv(( ffdd,, ffoorrmmaatt,, aapp )) + iinntt ffdd ;; + cchhaarr **ffoorrmmaatt ;; + vvaa__lliisstt aapp ;; + + iinntt SSddoonnee(( ffdd )) + iinntt ffdd ;; + + iinntt SSuunnddoo(( ffdd,, ttyyppee )) + iinntt ffdd ;; + iinntt ttyyppee ;; + + iinntt SSttiiee(( iiffdd,, ooffdd )) + iinntt iiffdd,, ooffdd ;; + + iinntt SSuunnttiiee(( ffdd )) + iinntt ffdd ;; + + + + 29 May 1992 1 + + + + + +SIO(3X) SIO(3X) + + + iinntt SSbbuuffttyyppee(( ffdd,, ttyyppee )) + iinntt ffdd,, ttyyppee ;; + + iinntt SSmmoorreeffddss(()) + + iinntt SSfflluusshh(( ffdd )) + iinntt ffdd ;; + + iinntt SScclloossee(( ffdd )) + iinntt ffdd ;; + + iinntt SSggeettcchhaarr(( ffdd )) + iinntt ffdd ;; + + iinntt SSppuuttcchhaarr(( ffdd,, cc )) + iinntt ffdd;; + cchhaarr cc ;; + + iinntt SSIIOOLLIINNEELLEENN(( ffdd )) + iinntt ffdd ;; + + iinntt SSIIOOMMAAXXLLIINNEELLEENN(( ffdd )) + iinntt ffdd ;; + +DDEESSCCRRIIPPTTIIOONN + The _S_I_O library provides support for _s_t_r_e_a_m I/O on file + descriptors. The first argument of every function or + macro is a file descriptor. The file descriptor may be + used either for input or for output, but not both. + Attempting to use a descriptor for both input and output + will cause the call for the latter use to fail. When you + are done with using a file descriptor, you should inform + _S_I_O by invoking SSddoonnee(()) (unless the program is about to + call _e_x_i_t_(_3_)). You can also use SSddoonnee(()) if you want to + perform a different type of operation on the same file + descriptor (e.g. first you were reading data from the file + descriptor and then you want to write some data). Another + possibility is to do stream I/O at different file offsets + by using SSddoonnee(()) before using llsseeeekk((22)) to move to a new + file offset. + + I/O operations on different file descriptors do not inter- + fere (unless the file descriptors refer to the same file, + in which case the results are undefined). + + For disk files, I/O always starts at the current file off- + set. If that offset is not a multiple of the preferred + block size for file system I/O, performance will not be + optimal (the preferred block size is determined from the + _s_t___b_l_k_s_i_z_e field in _s_t_r_u_c_t _s_t_a_t). For optimal perfor- + mance, it is recommended that no I/O operations (like + _r_e_a_d_(_2_) or _w_r_i_t_e_(_2_)) are applied to the file descriptor if + it is to be used by _S_I_O. + + + + + 29 May 1992 2 + + + + + +SIO(3X) SIO(3X) + + + Read I/O is either buffered, or is done using memory map- + ping whenever that is possible and appropriate. + + The library functions that do stream I/O resemble system + calls (for example SSrreeaadd(()) resembles _r_e_a_d_(_2_)) so that mod- + ifying a program that uses the system calls to use the _S_I_O + functions is easy (e.g. just replace _r_e_a_d_(_2_) with SSrreeaadd(()); + the function signatures as well as the return values are + exactly the same; also make sure to replace calls to + _c_l_o_s_e_(_2_) with SScclloossee(())). + + _S_I_O uses the underlying system calls _r_e_a_d_(_2_) and _w_r_i_t_e_(_2_) + to do I/O (except when reading files using memory map- + ping). These calls may be interrupted (i.e. they may + return -1 with _e_r_r_n_o set to EINTR). Such interruptions are + ignored by _S_I_O which simply reissues the system call (this + means that a _S_I_O call will never fail because the underly- + ing I/O system call was interrupted). + + SSrreeaadd(()) reads _n_b_y_t_e_s bytes from the stream associated with + file descriptor _f_d into the buffer pointed to by _b_u_f. + + SSggeettcc(()) reads a character from the stream associated with + file descriptor _f_d. It returns SSIIOO__EEOOFF if the end of file + has been reached. + + SSggeettcchhaarr(()) (a macro) performs exactly the same function as + SSggeettcc(()) but it is much faster. + + SSrrddlliinnee(()) reads a line from the stream associated with + file descriptor _f_d. The newline at the end of the line is + replaced by a NUL byte. Lines longer than the maximum line + length supported by _S_I_O will have characters deleted. + + SSIIOOLLIINNEELLEENN(()) (a macro) returns the length of the line + returned by the last call to SSrrddlliinnee(()) (the value returned + by SSIIOOLLIINNEELLEENN(()) is valid only after SSrrddlliinnee(()) and as long + as no other _S_I_O calls are performed on that file descrip- + tor). + + SSIIOOMMAAXXLLIINNEELLEENN(()) (a macro) returns the maximul line length + supported by _S_I_O for the file descriptor. As a side-effect + it initializes _f_d for input. + + SSffeettcchh(()) returns a pointer to data coming from the stream + associated with file descriptor _f_d. The amount of data + available is indicated by the _l_e_n_g_t_h argument. One possi- + ble use for this function is to copy files. + + SSwwrriittee(()) writes _n_b_y_t_e_s bytes to the stream associated with + file descriptor _f_d from the buffer pointed to by _b_u_f. + + SSppuuttcc(()) writes a single character to the stream associated + with file descriptor _f_d. + + + + 29 May 1992 3 + + + + + +SIO(3X) SIO(3X) + + + SSppuuttcchhaarr(()) (a macro) performs exactly the same function as + SSppuuttcc(()) but it is much faster. + + SSpprriinntt(()) imitates the behavior of printf(3) as defined in + the ANSI C Standard. There are some limitations. Check the + SSpprriinntt(()) man page for more information. + + SSpprriinnttvv(()) is the same as SSpprriinntt(()) except that it uses a + _v_a_r_a_r_g_s argument list. + + SSuunnddoo(()) returns the characters returned by the last call + to SSrrddlliinnee(()), SSggeettcc(()) or SSggeettcchhaarr(()) to the stream so that + they can be reread. The _t_y_p_e argument to SSuunnddoo(()) can be + SSIIOO__UUNNDDOO__LLIINNEE or SSIIOO__UUNNDDOO__CCHHAARR depending on whether the + call whose effect needs to be undone was SSrrddlliinnee(()) or + SSggeettcc(())/SSggeettcchhaarr(()) respectively. There is no check on + whether the last function invoked on _f_d was one of the + above and the results are undefined if there is no corre- + spondence between the _t_y_p_e and the last operation on _f_d. + (i.e. the result is undefined if you try SSIIOO__UUNNDDOO__CCHHAARR and + the last operation was not SSggeettcchhaarr(()) or SSggeettcc(())). + + SSttiiee(()) ties the file descriptor _i_f_d to the file descriptor + _o_f_d. This means that whenever a _r_e_a_d_(_2_) is done on _i_f_d, + it is preceded by a _w_r_i_t_e_(_2_) on _o_f_d. For filters it is + useful to do _S_t_i_e_( _0_, _1 _) to maximize concurrency. It is + also useful to do the same thing when you issue prompts to + the user and you want the user reply to appear on the same + line with the prompt. _i_f_d, _o_f_d will be initialized for + input, output respectively (if any of them is initialized, + it must be for the appropriate stream type (input or out- + put)). If _i_f_d was tied to another file descriptor, the + old tie is broken. + + SSuunnttiiee(()) undoes the effect of SSttiiee(()) for the specified + input file descriptor. + + SSbbuuffttyyppee(()) determines the buffering type for the output + stream associated with file descriptor _f_d. By default + output directed to terminals is line buffered, output + directed to file descriptor 2 (standard error) is + unbuffered and everything else is fully buffered. Possi- + ble values for the _t_y_p_e argument are + + SSIIOO__FFUULLLLBBUUFF for full buffering + + SSIIOO__LLIINNEEBBUUFF for line buffering + + SSIIOO__NNOOBBUUFF for no buffering + + SSmmoorreeffddss(()) should be used to inform SSIIOO that the number of + available file descriptors has been increased. SSIIOO uses an + array of internal stream descriptors which are indexed by + the file descriptor number. Some operating systems (ex. + + + + 29 May 1992 4 + + + + + +SIO(3X) SIO(3X) + + + SunOS 4.1[.x]) allow the number of available file descrip- + tors to vary. If that number is increased beyond its ini- + tial value SSIIOO needs to know in order to allocate more + stream descriptors. + + SSddoonnee(()) flushes any buffered output for _f_d and releases + the _S_I_O resources used. SSddoonnee(()) is useful in case the pro- + gram needs to reprocess the data of a file descriptor + (assuming the file descriptor corresponds to a file). The + program can call SSddoonnee(()), _l_s_e_e_k_(_2_) to the beginning of the + file and then proceed to reread the file. + + SSfflluusshh(()) causes any buffered stream output to be written + to the file descriptor. If its argument is the special + value SSIIOO__FFLLUUSSHH__AALLLL then all output streams will be + flushed. + + SScclloossee(()) closes a file descriptor used for stream I/O, + flushes any buffered output and releases the _S_I_O resources + used. + +EEXXAAMMPPLLEESS + The following code implements a (poor) substitute for the + tee command (it copies standard input to a file as well as + to standard output). + ##iinncclluuddee ""ssiioo..hh"" + mmaaiinn(( aarrggcc,, aarrggvv )) + iinntt aarrggcc ;; + cchhaarr **aarrggvv[[]] ;; + {{ + cchhaarr **ffiillee == ((aarrggcc >> 11)) ?? aarrggvv[[ 11 ]] :: ""tteeee..ffiillee"" ;; + iinntt ffdd == ccrreeaatt(( ffiillee,, 00664444 )) ;; + lloonngg lleennggtthh ;; + cchhaarr **ss ;; + wwhhiillee (( ss == SSffeettcchh(( 00,, &&lleennggtthh )) )) + {{ + SSwwrriittee(( 11,, ss,, lleennggtthh )) ;; + SSwwrriittee(( ffdd,, ss,, lleennggtthh )) ;; + }} + eexxiitt(( 00 )) ;; + }} + +RREETTUURRNN VVAALLUUEESS + SSrreeaadd(()) returns the number of bytes read on success (0 + means end-of-file) or SSIIOO__EERRRR on failure (_e_r_r_n_o is set to + indicate the error). + + SSggeettcc(()) returns the character read on success, SIO_EOF + when the end-of-file is reached, or SSIIOO__EERRRR on failure + (_e_r_r_n_o is set to indicate the error). + + SSrrddlliinnee(()) returns a pointer to the next line on success. + On failure or when the end-of-file is reached it returns + NULL. If the end-of-file is reached _e_r_r_n_o is set to 0, + + + + 29 May 1992 5 + + + + + +SIO(3X) SIO(3X) + + + otherwise it indicates the error. + + SSffeettcchh(()) returns a pointer to file data on success. (the + _l_e_n_g_t_h argument indicates how many bytes are available). + On failure or when the end-of-file is reached it returns + NULL. If the end-of-file is reached _e_r_r_n_o is set to 0, + otherwise it indicates the error. + + SSwwrriittee(()) returns the number of bytes written on success or + SSIIOO__EERRRR on failure (_e_r_r_n_o is set to indicate the error). + + SSppuuttcc(()) returns the character it was given as an argument + on success SSpprriinntt(()) returns the number of characters + printed on success or SSIIOO__EERRRR on failure (_e_r_r_n_o is set to + indicate the error). + + SSddoonnee(()) returns 00 on success or SSIIOO__EERRRR on failure (_e_r_r_n_o + is set to indicate the error). + + SSuunnddoo(()) returns 00 on success or SSIIOO__EERRRR on failure (_e_r_r_n_o + is set to indicate the error). + + SSttiiee(()) returns 00 on success or SSIIOO__EERRRR on failure (_e_r_r_n_o + is set to indicate the error). + + SSuunnttiiee(()) returns 00 on success or SSIIOO__EERRRR on failure (_e_r_r_n_o + is set to EEBBAADDFF if there was no tied file descriptor). + + SSbbuuffttyyppee(()) returns 00 on success or SSIIOO__EERRRR on failure + (_e_r_r_n_o is set to EEBBAADDFF if this is not an output stream or + to EEIINNVVAALL if an unknown _t_y_p_e is specified). + + SSmmoorreeffddss(()) returns 00 on success or SSIIOO__EERRRR on failure + (because of lack of memory). + + SSfflluusshh(()) returns 00 on success or SSIIOO__EERRRR on failure (_e_r_r_n_o + is set to indicate the error). + + SScclloossee(()) returns 00 on success or SSIIOO__EERRRR on failure (_e_r_r_n_o + is set to indicate the error). + + SSggeettcchhaarr(()) returns the character read on success, SIO_EOF + when the end-of-file is reached, or SSIIOO__EERRRR on failure + (_e_r_r_n_o is set to indicate the error). + + SSppuuttcchhaarr(()) returns the character it was given as an argu- + ment on success or SSIIOO__EERRRR on failure (_e_r_r_n_o is set to + indicate the error). + + SSIIOOLLIINNEELLEENN(()) returns the length of the last line read by + SSrrddlliinnee(()). + + SSIIOOMMAAXXLLIINNEELLEENN(()) returns the length of the longest line + supported by _S_I_O on success or SSIIOO__EERRRR on failure (_e_r_r_n_o + + + + 29 May 1992 6 + + + + + +SIO(3X) SIO(3X) + + + is set to indicate the error). + + Attempting a read operation on a descriptor opened for + writing or vice versa will cause the operation to fail + with _e_r_r_n_o set to EEBBAADDFF. + + The first _S_I_O operation on a descriptor must be a read or + write operation. It cannot be a control operation (like + SSfflluusshh(())). Such an operation will fail with _e_r_r_n_o set to + EEBBAADDFF. + + + NNOOTTEE 11:: SSttiiee(()) is an input/output operation for the + respective file descriptors, not a control + operation. SSuunnttiiee(()) is a control operation. + + NNOOTTEE 22:: SSIIOO__EERRRR is defined to be --11. + +SSEEEE AALLSSOO + Sprint(3) + +BBUUGGSS + If the operating system does not provide for invocation of + a finalization function upon exit, the program will have + to explicitly flush all output streams. The following + operating systems provide such a facility: SunOS 4.x, + Ultrix 4.x, SunOS 5.x + + Socket file descriptors can be used for input as well as + output but SSIIOO does not support this. + + The current implementation will not try to use memory map- + ping to read a file if the file offset is not 0 (it will + use buffered I/O instead). + + Pointers returned by SSffeettcchh(()) point to read-only memory. + Attempting to modify this memory will result in a segmen- + tation violation. + + + + + + + + + + + + + + + + + + + + 29 May 1992 7 + + Index: ossp-pkg/sio/BRAINSTORM/postfix-vstream.txt RCS File: /v/ossp/cvs/ossp-pkg/sio/BRAINSTORM/postfix-vstream.txt,v co -q -kk -p'1.1' '/v/ossp/cvs/ossp-pkg/sio/BRAINSTORM/postfix-vstream.txt,v' | diff -u /dev/null - -L'ossp-pkg/sio/BRAINSTORM/postfix-vstream.txt' 2>/dev/null --- ossp-pkg/sio/BRAINSTORM/postfix-vstream.txt +++ - 2024-05-15 09:15:08.644161789 +0200 @@ -0,0 +1,396 @@ + + + +VSTREAM(3) VSTREAM(3) + + +NNAAMMEE + vstream - light-weight buffered I/O package + +SSYYNNOOPPSSIISS + #include + + VSTREAM *vstream_fopen(path, flags, mode) + char *path; + int flags; + int mode; + + VSTREAM *vstream_fdopen(fd, flags) + int fd; + int flags; + + int vstream_fclose(stream) + VSTREAM *stream; + + VSTREAM *vstream_printf(format, ...) + char *format; + + VSTREAM *vstream_fprintf(stream, format, ...) + VSTREAM *stream; + char *format; + + int VSTREAM_GETC(stream) + VSTREAM *stream; + + int VSTREAM_PUTC(ch, stream) + int ch; + + int VSTREAM_GETCHAR(void) + + int VSTREAM_PUTCHAR(ch) + int ch; + + int vstream_ungetc(stream, ch) + VSTREAM *stream; + int ch; + + int vstream_fputs(str, stream) + char *str; + VSTREAM *stream; + + long vstream_ftell(stream) + VSTREAM *stream; + + long vstream_fseek(stream, offset, whence) + VSTREAM *stream; + long offset; + int whence; + + int vstream_fflush(stream) + VSTREAM *stream; + + + + 1 + + + + + +VSTREAM(3) VSTREAM(3) + + + int vstream_fread(stream, buf, len) + VSTREAM *stream; + char *buf; + int len; + + int vstream_fwrite(stream, buf, len) + VSTREAM *stream; + char *buf; + int len; + + void vstream_control(stream, name, ...) + VSTREAM *stream; + int name; + + int vstream_fileno(stream) + VSTREAM *stream; + + int vstream_ferror(stream) + VSTREAM *stream; + + int vstream_feof(stream) + VSTREAM *stream; + + int vstream_clearerr(stream) + VSTREAM *stream; + + char *VSTREAM_PATH(stream) + VSTREAM *stream; + + char *vstream_vfprintf(vp, format, ap) + char *format; + va_list *ap; + + int vstream_peek(stream) + VSTREAM *stream; + +DDEESSCCRRIIPPTTIIOONN + The _v_s_t_r_e_a_m module implements light-weight buffered I/O + similar to the standard I/O routines. + + The interface is implemented in terms of VSTREAM structure + pointers, also called streams. For convenience, three + streams are predefined: VSTREAM_IN, VSTREAM_OUT, and + VSTREAM_ERR. These streams are connected to the standard + input, output and error file descriptors, respectively. + + Although the interface is patterned after the standard I/O + library, there are some major differences: + + +o File descriptors are not limited to the range + 0..255. This was reason #1 to write these routines + in the first place. + + +o The application can switch between reading and + + + + 2 + + + + + +VSTREAM(3) VSTREAM(3) + + + writing on the same stream without having to per- + form a flush or seek operation, and can change + write position without having to flush. This was + reason #2. Upon position or direction change, + unread input is discarded, and unwritten output is + flushed automatically. Exception: with double- + buffered streams, unread input is not discarded + upon change of I/O direction, and output flushing + is delayed until the read buffer must be refilled. + + +o A bidirectional stream can read and write with the + same buffer and file descriptor, or it can have + separate read/write buffers and/or file descrip- + tors. + + +o No automatic flushing of VSTREAM_OUT upon program + exit, or of VSTREAM_ERR at any time. No unbuffered + or line buffered modes. This functionality may be + added when it is really needed. + + vstream_fopen() opens the named file and associates a + buffered stream with it. The _p_a_t_h, _f_l_a_g_s and _m_o_d_e argu- + ments are passed on to the open(2) routine. The result is + a null pointer in case of problems. The _p_a_t_h argument is + copied and can be looked up with VSTREAM_PATH(). + + vstream_fdopen() takes an open file and associates a + buffered stream with it. The _f_l_a_g_s argument specifies how + the file was opened. vstream_fdopen() either succeeds or + never returns. + + vstream_fclose() closes the named buffered stream. The + result is 0 in case of success, VSTREAM_EOF in case of + problems. + + vstream_fprintf() formats its arguments according to the + _f_o_r_m_a_t argument and writes the result to the named stream. + The result is the stream argument. It understands the s, + c, d, u, o, x, X, e, f and g format types, the l modifier, + field width and precision, sign, and padding with zeros or + spaces. In addition, vstream_fprintf() recognizes the %m + format specifier and expands it to the error message cor- + responding to the current value of the global _e_r_r_n_o vari- + able. + + vstream_printf() performs formatted output to the standard + output stream. + + VSTREAM_GETC() reads the next character from the named + stream. The result is VSTREAM_EOF when end-of-file is + reached or if a read error was detected. VSTREAM_GETC() is + an unsafe macro that evaluates some arguments more than + once. + + + + + 3 + + + + + +VSTREAM(3) VSTREAM(3) + + + VSTREAM_GETCHAR() is an alias for + VSTREAM_GETC(VSTREAM_IN). + + VSTREAM_PUTC() appends the specified character to the + specified stream. The result is the stored character, or + VSTREAM_EOF in case of problems. VSTREAM_PUTC() is an + unsafe macro that evaluates some arguments more than once. + + VSTREAM_PUTCHAR(c) is an alias for VSTREAM_PUTC(c, + VSTREAM_OUT). + + vstream_unget() pushes back a character onto the specified + stream and returns the character, or VSTREAM_EOF in case + of problems. It is an error to push back before reading + (or immediately after changing the stream offset via + vstream_fseek()). Upon successful return, vstream_unget() + clears the end-of-file stream flag. + + vstream_fputs() appends the given null-terminated string + to the specified buffered stream. The result is 0 in case + of success, VSTREAM_EOF in case of problems. + + vstream_ftell() returns the file offset for the specified + stream, -1 if the stream is connected to a non-seekable + file. + + vstream_fseek() changes the file position for the next + read or write operation. Unwritten output is flushed. With + unidirectional streams, unread input is discarded. The + _o_f_f_s_e_t argument specifies the file position from the + beginning of the file (_w_h_e_n_c_e is SEEK_SET), from the cur- + rent file position (_w_h_e_n_c_e is SEEK_CUR), or from the file + end (SEEK_END). The result value is the file offset from + the beginning of the file, -1 in case of problems. + + vstream_fflush() flushes unwritten data to a file that was + opened in read-write or write-only mode. vstream_fflush() + returns 0 in case of success, VSTREAM_EOF in case of prob- + lems. It is an error to flush a read-only stream. + + vstream_fread() and vstream_fwrite() perform unformatted + I/O on the named stream. The result value is the number of + bytes transferred. A short count is returned in case of + end-of-file or error conditions. + + vstream_control() allows the user to fine tune the behav- + ior of the specified stream. The arguments are a list of + (name, value) pairs, terminated with VSTREAM_CTL_END. The + following lists the names and the types of the correspond- + ing value arguments. + + VSTREAM_CTL_READ_FN (int (*)(int, void *, unsigned)) + The argument specifies an alternative for the + read(2) function, for example, a read function that + + + + 4 + + + + + +VSTREAM(3) VSTREAM(3) + + + enforces a time limit. + + VSTREAM_CTL_WRITE_FN (int (*)(int, void *, unsigned)) + The argument specifies an alternative for the + write(2) function, for example, a write function + that enforces a time limit. + + VSTREAM_CTL_PATH (char *) + Updates the stored pathname of the specified + stream. The pathname is copied. + + VSTREAM_CTL_DOUBLE (no value) + Use separate buffers for reading and for writing. + This prevents unread input from being discarded + upon change of I/O direction. + + VSTREAM_CTL_READ_FD (int) + The argument specifies the file descriptor to be + used for reading. This feature is limited to dou- + ble-buffered streams, and makes the stream non- + seekable. + + VSTREAM_CTL_WRITE_FD (int) + The argument specifies the file descriptor to be + used for writing. This feature is limited to dou- + ble-buffered streams, and makes the stream non- + seekable. + + VSTREAM_CTL_WAITPID_FN (int (*)(pid_t, WAIT_STATUS_T *, + int)) + A pointer to function that behaves like waitpid(). + This information is used by the vstream_pclose() + routine. + + vstream_fileno() gives access to the file handle associ- + ated with a buffered stream. With streams that have sepa- + rate read/write file descriptors, the result is the cur- + rent descriptor. + + VSTREAM_PATH() is an unsafe macro that returns the name + stored with vstream_fopen() or with vstream_control(). The + macro is unsafe because it evaluates some arguments more + than once. + + vstream_ferror() (vstream_feof()) returns non-zero when a + previous operation on the specified stream caused an error + (end-of-file) condition. + + vstream_clearerr() resets the error and end-of-file indi- + cation of specified stream, and returns no useful result. + + vstream_vfprintf() provides an alternate interface for + formatting an argument list according to a format string. + + + + + 5 + + + + + +VSTREAM(3) VSTREAM(3) + + + vstream_peek() returns the number of characters that can + be read from the named stream without refilling the read + buffer. + +DDIIAAGGNNOOSSTTIICCSS + Panics: interface violations. Fatal errors: out of memory. + +SSEEEE AALLSSOO + vbuf_print(3) formatting engine + +BBUUGGSS + Should use mmap() on reasonable systems. + +LLIICCEENNSSEE + The Secure Mailer license must be distributed with this + software. + +AAUUTTHHOORR((SS)) + Wietse Venema + IBM T.J. Watson Research + P.O. Box 704 + Yorktown Heights, NY 10598, USA + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 6 + + Index: ossp-pkg/sio/BRAINSTORM/stash-buf.txt RCS File: /v/ossp/cvs/ossp-pkg/sio/BRAINSTORM/stash-buf.txt,v co -q -kk -p'1.1' '/v/ossp/cvs/ossp-pkg/sio/BRAINSTORM/stash-buf.txt,v' | diff -u /dev/null - -L'ossp-pkg/sio/BRAINSTORM/stash-buf.txt' 2>/dev/null --- ossp-pkg/sio/BRAINSTORM/stash-buf.txt +++ - 2024-05-15 09:15:08.646790934 +0200 @@ -0,0 +1,652 @@ +/* -*- mode: c ; c-file-style: "canonware-c-style" -*- + **************************************************************************** + * + * Copyright (C) 1996-1999 Jason Evans . + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice(s), this list of conditions and the following disclaimer as + * the first lines of this file unmodified other than the possible + * addition of one or more copyright notices. + * 2. Redistributions in binary form must reproduce the above copyright + * notice(s), this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER(S) ``AS IS'' AND ANY + * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER(S) BE + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR + * BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, + * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE + * OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, + * EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + **************************************************************************** + * + * Version: s19990524a + * + * <<< Description >>> + * + * The buf and bufc classes implement a buffer abstraction. These classes are + * designed specifically to handle streaming and transparent extensible + * buffering of data for applications such as socket programs. The main + * features include: + * + * - Dynamically extensible and contractible buffering. + * - Internal reference counting, which avoids copying between buf's, and allows + * for compact usage of memory buffers. + * - 8, 32, and 64 bit read functions for arbitrary byte offsets (within the + * valid data range) within buf's. + * - 8, 32, and 64 bit, and string, write/append functions for arbitrary byte + * offsets. + * - Easy ability to use with readv() and writev(). + * + ****************************************************************************/ + +/* Pseudo-opaque typedefs. */ +typedef struct cw_buf_s cw_buf_t; +typedef struct cw_bufc_s cw_bufc_t; + +/* Opaque typedef. */ +typedef struct cw_bufel_s cw_bufel_t; + +/* The following data types should be considered opaque. */ +struct cw_bufc_s +{ +#if (defined(_LIBSTASH_DBG) || defined(_LIBSTASH_DEBUG)) + cw_uint32_t magic; +#endif +#ifdef _CW_REENTRANT + cw_mtx_t lock; +#endif + void (*dealloc_func)(void *, void *); + void * dealloc_arg; + void (*buffer_dealloc_func)(void *, void *); + void * buffer_dealloc_arg; + cw_uint32_t ref_count; + cw_bool_t is_writeable; + cw_uint32_t buf_size; + cw_uint8_t * buf; +}; + +struct cw_buf_s +{ +#if (defined(_LIBSTASH_DBG) || defined(_LIBSTASH_DEBUG)) + cw_uint32_t magic; +#endif + cw_bool_t is_malloced; +#ifdef _CW_REENTRANT + cw_bool_t is_threadsafe; + cw_mtx_t lock; +#endif + cw_uint32_t size; + + cw_uint32_t array_size; + cw_uint32_t array_num_valid; + cw_uint32_t array_start; + cw_uint32_t array_end; + cw_bool_t is_cumulative_valid; + cw_bool_t is_cached_bufel_valid; + cw_uint32_t cached_bufel; + cw_bufel_t * bufel_array; + cw_uint32_t * cumulative_index; + struct iovec * iov; +}; + +/**************************************************************************** + * + * <<< Input(s) >>> + * + * a_buf : Pointer to space for a buf, or NULL. + * + * a_is_thread_safe : FALSE == not thread safe, TRUE == thread safe. + * + * <<< Output(s) >>> + * + * retval : Pointer to a buf, or NULL. + * NULL : Memory allocation error. + * + * <<< Description >>> + * + * Constructor. + * + ****************************************************************************/ +#ifdef _CW_REENTRANT +cw_buf_t * +buf_new(cw_buf_t * a_buf, cw_bool_t a_is_threadsafe); +#else +cw_buf_t * +buf_new(cw_buf_t * a_buf); +#endif + +/**************************************************************************** + * + * <<< Input(s) >>> + * + * a_buf : Pointer to a buf. + * + * <<< Output(s) >>> + * + * None. + * + * <<< Description >>> + * + * Destructor. + * + ****************************************************************************/ +void +buf_delete(cw_buf_t * a_buf); + +/**************************************************************************** + * + * <<< Input(s) >>> + * + * a_buf : Pointer to a buf. + * + * a_prefix : Pointer to a string that represents a string to be prefixed to + * each line of output. + * + * <<< Output(s) >>> + * + * None. + * + * <<< Description >>> + * + * Dump the internal state of a_buf to cw_g_log. + * + ****************************************************************************/ +void +buf_dump(cw_buf_t * a_buf, const char * a_prefix); + +/**************************************************************************** + * + * <<< Input(s) >>> + * + * a_buf : Pointer to a buf. + * + * <<< Output(s) >>> + * + * retval : Number of bytes of valid data. + * + * <<< Description >>> + * + * Return the amount of valid data in bytes. + * + ****************************************************************************/ +cw_uint32_t +buf_get_size(cw_buf_t * a_buf); + +/**************************************************************************** + * + * <<< Input(s) >>> + * + * a_buf : Pointer to a buf. + * + * <<< Output(s) >>> + * + * retval : Number of bufel's in a_buf (same as iovec count in buf_get_iovec()). + * + * <<< Description >>> + * + * Return the number of bufel's in a_buf. + * + ****************************************************************************/ +cw_uint32_t +buf_get_num_bufels(cw_buf_t * a_buf); + +/**************************************************************************** + * + * <<< Input(s) >>> + * + * a_buf : Pointer to a buf. + * + * a_max_data : Maximum number of bytes of space to include in the iovec. + * + * a_is_sys_iovec : If TRUE, limit *r_iovec_count to the maximum iovec count + * supported by this system for readv()/writev(). + * + * a_iovec_count : Pointer to an int. + * + * <<< Output(s) >>> + * + * retval : Pointer to an iovec array that represents the internal data buffers + * in a_buf. + * + * *r_iovec_count : Number of valid iovec structures in retval. + * + * <<< Description >>> + * + * Build an iovec array that represents the valid data in a_buf's internal + * buffers (up to a_max_data bytes) and return a pointer to it. + * + ****************************************************************************/ +const struct iovec * +buf_get_iovec(cw_buf_t * a_buf, cw_uint32_t a_max_data, + cw_bool_t a_is_sys_iovec, int * r_iovec_count); + +/**************************************************************************** + * + * <<< Input(s) >>> + * + * a_a : Pointer to a buf. + * + * a_b : Pointer to a buf. + * + * a_preserve : If TRUE, preserve a_b (don't modify it). If FALSE, release the + * data in a_b after catenating a_b to a_a. + * + * <<< Output(s) >>> + * + * retval : FALSE == success, TRUE == error. + * TRUE : Memory allocation error. + * + * <<< Description >>> + * + * Catenate two bufs. a_b is left unmodified if a_preserve is TRUE. + * + ****************************************************************************/ +cw_bool_t +buf_catenate_buf(cw_buf_t * a_a, cw_buf_t * a_b, cw_bool_t a_preserve); + +/**************************************************************************** + * + * <<< Input(s) >>> + * + * a_a : Pointer to a buf. + * + * a_b : Pointer to a buf. + * + * a_offset : Offset at which to split a_b. + * + * <<< Output(s) >>> + * + * retval : FALSE == success, TRUE == error. + * TRUE : Memory allocation error. a_a and a_b are returned to their + * original states. + * + * <<< Description >>> + * + * Split a_b at offset a_offset. Append the data before a_offset to a_a, and + * leave the remainder in a_b. + * + ****************************************************************************/ +cw_bool_t +buf_split(cw_buf_t * a_a, cw_buf_t * a_b, cw_uint32_t a_offset); + +/**************************************************************************** + * + * <<< Input(s) >>> + * + * a_buf : Pointer to a buf. + * + * a_bufel : Pointer to a bufel. + * + * <<< Output(s) >>> + * + * retval : FALSE == success, TRUE == error. + * TRUE : Memory allocation error. a_buf is still valid. + * + * <<< Description >>> + * + * Prepend the data from a_bufel to a_buf. a_bufel is not modified. + * + ****************************************************************************/ +cw_bool_t +buf_prepend_bufel(cw_buf_t * a_buf, cw_bufel_t * a_bufel); + + +/**************************************************************************** + * + * <<< Input(s) >>> + * + * a_buf : Pointer to a buf. + * + * a_bufc : Pointer to a bufc. + * + * a_beg_offset : Offset of first valid byte in a_bufc's memory buffer. + * + * a_end_offset : Offset of first byte past the valid range of bytes in a_bufc's + * memory buffer. + * + * <<< Output(s) >>> + * + * retval : FALSE == success, TRUE == error. + * TRUE : Memory allocation error. a_buf is still valid. + * + * <<< Description >>> + * + * Prepend a_bufc, bytes a_beg_offset .. (a_end_offset - 1) to a_buf. + * + ****************************************************************************/ +cw_bool_t +buf_prepend_bufc(cw_buf_t * a_buf, cw_bufc_t * a_bufc, + cw_uint32_t a_beg_offset, cw_uint32_t a_end_offset); + +/**************************************************************************** + * + * <<< Input(s) >>> + * + * a_buf : Pointer to a buf. + * + * a_bufc : Pointer to a bufc. + * + * a_beg_offset : Offset of first valid byte in a_bufc's memory buffer. + * + * a_end_offset : Offset of first byte past the valid range of bytes in a_bufc's + * memory buffer. + * + * <<< Output(s) >>> + * + * retval : FALSE == success, TRUE == error. + * TRUE : Memory allocation error. a_buf is still valid. + * + * <<< Description >>> + * + * Append a_bufc, bytes a_beg_offset .. (a_end_offset - 1) to a_buf. + * + ****************************************************************************/ +cw_bool_t +buf_append_bufc(cw_buf_t * a_buf, cw_bufc_t * a_bufc, + cw_uint32_t a_beg_offset, cw_uint32_t a_end_offset); + +/**************************************************************************** + * + * <<< Input(s) >>> + * + * a_buf : Pointer to a buf. + * + * a_bufel : Pointer to a bufel. + * + * <<< Output(s) >>> + * + * retval : FALSE == success, TRUE == error. + * TRUE : Memory allocation error. a_buf is still valid. + * + * <<< Description >>> + * + * Append the data from a_bufel to a_buf. a_bufel is not modified. + * + ****************************************************************************/ +cw_bool_t +buf_append_bufel(cw_buf_t * a_buf, cw_bufel_t * a_bufel); + +/**************************************************************************** + * + * <<< Input(s) >>> + * + * a_buf : Pointer to a buf. + * + * a_amount : Number of bytes of data to release from the head of a_buf. + * + * <<< Output(s) >>> + * + * retval : FALSE == success, TRUE == error. + * + * <<< Description >>> + * + * Release a_amount bytes from the head of a_buf. + * + ****************************************************************************/ +cw_bool_t +buf_release_head_data(cw_buf_t * a_buf, cw_uint32_t a_amount); + +/**************************************************************************** + * + * <<< Input(s) >>> + * + * a_buf : Pointer to a buf. + * + * a_amount : Number of bytes of data to release from the tail of a_buf. + * + * <<< Output(s) >>> + * + * retval : FALSE == success, TRUE == error. + * + * <<< Description >>> + * + * Release a_amount bytes from the tail of a_buf. + * + ****************************************************************************/ +cw_bool_t +buf_release_tail_data(cw_buf_t * a_buf, cw_uint32_t a_amount); + +/**************************************************************************** + * + * <<< Input(s) >>> + * + * a_buf : Pointer to a buf. + * + * a_offset : Offset in bytes of uint8 to return. + * + * <<< Output(s) >>> + * + * retval : Value of the uint8 at offset a_offset in a_buf. + * + * <<< Description >>> + * + * Return the uint8 at offset a_offset. + * + ****************************************************************************/ +cw_uint8_t +buf_get_uint8(cw_buf_t * a_buf, cw_uint32_t a_offset); + +/**************************************************************************** + * + * <<< Input(s) >>> + * + * a_buf : Pointer to a buf. + * + * a_offset : Offset in bytes of uint32 to return. + * + * <<< Output(s) >>> + * + * retval : Value of the uint32 at offset a_offset in a_buf. + * + * <<< Description >>> + * + * Return the uint32 at offset a_offset. + * + ****************************************************************************/ +cw_uint32_t +buf_get_uint32(cw_buf_t * a_buf, cw_uint32_t a_offset); + +/**************************************************************************** + * + * <<< Input(s) >>> + * + * a_buf : Pointer to a buf. + * + * a_offset : Offset in bytes of uint64 to return. + * + * <<< Output(s) >>> + * + * retval : Value of the uint64 at offset a_offset in a_buf. + * + * <<< Description >>> + * + * Return the uint64 at offset a_offset. + * + ****************************************************************************/ +cw_uint64_t +buf_get_uint64(cw_buf_t * a_buf, cw_uint32_t a_offset); + +/**************************************************************************** + * + * <<< Input(s) >>> + * + * a_buf : Pointer to a buf. + * + * a_offset : Offset in bytes of data to set. (a_offset <= buf_get_size(a_buf)) + * + * a_val : Value to set data at a_offset to. + * + * <<< Output(s) >>> + * + * retval : FALSE == success, TRUE == error. + * TRUE : memory allocation error. + * + * <<< Description >>> + * + * Set the uint8 at a_offset to a_val. + * + ****************************************************************************/ +cw_bool_t +buf_set_uint8(cw_buf_t * a_buf, cw_uint32_t a_offset, cw_uint8_t a_val); + +/**************************************************************************** + * + * <<< Input(s) >>> + * + * a_buf : Pointer to a buf. + * + * a_offset : Offset in bytes of data to set. (a_offset <= buf_get_size(a_buf)) + * + * a_val : Value to set data at a_offset to. + * + * <<< Output(s) >>> + * + * retval : FALSE == success, TRUE == error. + * TRUE : memory allocation error. + * + * <<< Description >>> + * + * Set the uint32 at a_offset to a_val. + * + ****************************************************************************/ +cw_bool_t +buf_set_uint32(cw_buf_t * a_buf, cw_uint32_t a_offset, cw_uint32_t a_val); + +/**************************************************************************** + * + * <<< Input(s) >>> + * + * a_buf : Pointer to a buf. + * + * a_offset : Offset in bytes of data to set. (a_offset <= buf_get_size(a_buf)) + * + * a_val : Value to set data at a_offset to. + * + * <<< Output(s) >>> + * + * retval : FALSE == success, TRUE == error. + * TRUE : memory allocation error. + * + * <<< Description >>> + * + * Set the uint64 at a_offset to a_val. + * + ****************************************************************************/ +cw_bool_t +buf_set_uint64(cw_buf_t * a_buf, cw_uint32_t a_offset, cw_uint64_t a_val); + +/**************************************************************************** + * + * <<< Input(s) >>> + * + * a_buf : Pointer to a buf. + * + * a_offset : Offset in bytes of data to set. (a_offset <= buf_get_size(a_buf)) + * + * a_length : Number of bytes to copy from a_val. + * + * a_val : Value to set data at a_offset to. + * + * a_is_writeable : FALSE == non-writeable buffer, TRUE == writeable buffer. + * + * <<< Output(s) >>> + * + * retval : FALSE == success, TRUE == error. + * TRUE : memory allocation error. + * + * <<< Description >>> + * + * Copy a_offset bytes from a_val to a_buf at offset a_offset. + * + ****************************************************************************/ +cw_bool_t +buf_set_range(cw_buf_t * a_buf, cw_uint32_t a_offset, cw_uint32_t a_length, + cw_uint8_t * a_val, cw_bool_t a_is_writeable); + +/**************************************************************************** + * + * <<< Input(s) >>> + * + * a_bufc : Pointer to space for a bufc, or NULL. + * + * a_dealloc_func : Pointer to a deallocation function for a_bufc, or NULL. + * Ignored if a_bufc == NULL. + * + * a_dealloc_arg : First argument to a_dealloc_func. + * + * <<< Output(s) >>> + * + * retval : Pointer to an initialized bufc, or NULL. + * NULL : Memory allocation error. + * + * <<< Description >>> + * + * Constructor. The return value should be used in a call to bufc_set_buffer(), + * then bufel_set_bufc(). bufc_delete() can be called at any time up until the + * call to bufel_set_bufc() to deallocate, but should not be called thereafter. + * + ****************************************************************************/ +cw_bufc_t * +bufc_new(cw_bufc_t * a_bufc, + void (*a_dealloc_func)(void * dealloc_arg, void * bufel), + void * a_dealloc_arg); + +/**************************************************************************** + * + * <<< Input(s) >>> + * + * a_bufc : Pointer to a bufc. + * + * <<< Output(s) >>> + * + * None. + * + * <<< Description >>> + * + * Destructor. Only call this if a_bufc was never used in a call to + * bufel_set_bufc(). + * + ****************************************************************************/ +void +bufc_delete(cw_bufc_t * a_bufc); + +/**************************************************************************** + * + * <<< Input(s) >>> + * + * a_bufc : Pointer to a bufc. + * + * a_buffer : Pointer to a buffer. + * + * a_size : Size of *a_buffer. + * + * a_dealloc_func : Pointer to a deallocation function for a_buffer, or NULL. + * + * a_dealloc_arg : First argument to a_dealloc_func. + * + * <<< Output(s) >>> + * + * None. + * + * <<< Description >>> + * + * Set a_bufc's internal data buffer to a_buffer, with size a_size, and + * deallocation function a_dealloc_func(a_dealloc_arg, a_buffer). + * + ****************************************************************************/ +void +bufc_set_buffer(cw_bufc_t * a_bufc, void * a_buffer, cw_uint32_t a_size, + cw_bool_t a_is_writeable, + void (*a_dealloc_func)(void * dealloc_arg, void * buffer), + void * a_dealloc_arg); Index: ossp-pkg/sio/BRAINSTORM/substdio.txt RCS File: /v/ossp/cvs/ossp-pkg/sio/BRAINSTORM/substdio.txt,v co -q -kk -p'1.1' '/v/ossp/cvs/ossp-pkg/sio/BRAINSTORM/substdio.txt,v' | diff -u /dev/null - -L'ossp-pkg/sio/BRAINSTORM/substdio.txt' 2>/dev/null --- ossp-pkg/sio/BRAINSTORM/substdio.txt +++ - 2024-05-15 09:15:08.649553064 +0200 @@ -0,0 +1,396 @@ + + + +substdio(3) substdio(3) + + +NNAAMMEE + substdio - the Sub-Standard Input/Output Library + +SSYYNNTTAAXX + ##iinncclluuddee <> + + void ssuubbssttddiioo__ffddbbuuff(&_s,_o_p,_f_d,_b_u_f,_l_e_n); + + int ssuubbssttddiioo__ffiilleennoo(&_s); + + substdio _s; + int (*_o_p)(); + int _f_d; + char *_b_u_f; + int _l_e_n; + +DDEESSCCRRIIPPTTIIOONN + ssuubbssttddiioo is the Sub-Standard I/O Library. ssuubbssttddiioo con- + tains only a few of the features of stdio; it is a fast, + lightweight, low-level library, suitable for use as a com- + ponent of higher-level I/O libraries. + + The point of ssuubbssttddiioo is to provide buffered I/O. The + basic object in ssuubbssttddiioo is the ssuubbssttddiioo structure; a ssuubb-- + ssttddiioo variable stores an operation, a descriptor, and a + pointer into a buffer of some nonzero length. The ssuubbsstt-- + ddiioo operations read data from the buffer, filling the + buffer as necessary using the operation on the descriptor, + or write data to the buffer, flushing the buffer as neces- + sary using the operation on the descriptor. Input and + output operations cannot be mixed. + + ssuubbssttddiioo__ffddbbuuff initializes a ssuubbssttddiioo variable. The oper- + ation is _o_p. The descriptor is _f_d. The buffer is _b_u_f, an + array of _l_e_n chars. + + _o_p will be called as _o_p(_f_d,_x,_n). Here _x is a pointer to + an array of characters of length _n; _o_p must read some + characters from _f_d to that array, or write some characters + to _f_d from that array. The return value from _o_p must be + the number of characters read or written. 0 characters + read means end of input; 0 characters written means that + the write operation should be tried again immediately. On + error, _o_p must return -1, setting eerrrrnnoo appropriately, + without reading or writing anything. Most errors are + returned directly to the ssuubbssttddiioo caller, but an error of + eerrrroorr__iinnttrr means that the operation should be tried again + immediately. + + There is a SSUUBBSSTTDDIIOO__FFDDBBUUFF macro that can be used to stati- + cally initialize a ssuubbssttddiioo variable: + + substdio s = SUBSTDIO_FDBUF(op,fd,buf,len); + + + + + 1 + + + + + +substdio(3) substdio(3) + + + ssuubbssttddiioo__ffiilleennoo returns the descriptor for an initialized + ssuubbssttddiioo variable. + +SSEEEE AALLSSOO + substdio_in(3), substdio_out(3), substdio_copy(3), + error(3) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 2 + + + + + +substdio_in(3) substdio_in(3) + + +NNAAMMEE + substdio_in - substdio input routines + +SSYYNNTTAAXX + ##iinncclluuddee <> + + int ssuubbssttddiioo__ggeett(&_s,_t_o,_l_e_n); + + int ssuubbssttddiioo__bbggeett(&_s,_t_o,_l_e_n); + + int ssuubbssttddiioo__ffeeeedd(&_s); + + char *ssuubbssttddiioo__ppeeeekk(&_s); + + void ssuubbssttddiioo__sseeeekk(&_s,_l_e_n); + + substdio _s; + char *_t_o; + int _l_e_n; + +DDEESSCCRRIIPPTTIIOONN + ssuubbssttddiioo__ggeett reads at most _l_e_n characters from _s into the + character array _t_o. It returns the number of characters + read, 0 for end of file, or -1 for error, setting eerrrrnnoo + appropriately. + + ssuubbssttddiioo__bbggeett has the same function as ssuubbssttddiioo__ggeett. The + difference is what happens when there is no buffered data + and _l_e_n exceeds the buffer size: ssuubbssttddiioo__ggeett tries to + read _l_e_n characters, whereas ssuubbssttddiioo__bbggeett tries to read + one buffer of characters. In some cases ssuubbssttddiioo__bbggeett + will be more efficient than ssuubbssttddiioo__ggeett. + + ssuubbssttddiioo__ffeeeedd makes sure that there is buffered data, so + that the next ssuubbssttddiioo__ggeett or ssuubbssttddiioo__bbggeett will succeed. + If the buffer is empty, ssuubbssttddiioo__ffeeeedd tries to fill it; it + returns 0 for end of file, -1 for error, or the number of + buffered characters on success. If the buffer already had + data, ssuubbssttddiioo__ffeeeedd leaves it alone and returns the number + of buffered characters. + + ssuubbssttddiioo__ppeeeekk returns a pointer to the buffered data. + + ssuubbssttddiioo__sseeeekk throws away _l_e_n buffered characters, as if + they had been read. _l_e_n must be at least 0 and at most + the amount of buffered data. + + The ssuubbssttddiioo__PPEEEEKK and ssuubbssttddiioo__SSEEEEKK macros behave the same + way as ssuubbssttddiioo__ppeeeekk and ssuubbssttddiioo__sseeeekk but may evaluate + their arguments several times. + + The point of ssuubbssttddiioo__ppeeeekk and ssuubbssttddiioo__sseeeekk is to read + data without unnecessary copies. Sample code: + + + + + 1 + + + + + +substdio_in(3) substdio_in(3) + + + for (;;) { + n = substdio_feed(s); + if (n <= 0) return n; + x = substdio_PEEK(s); + handle(x,n); + substdio_SEEK(s,n); + } + + The SSUUBBSSTTDDIIOO__IINNSSIIZZEE macro is defined as a reasonably large + input buffer size for ssuubbssttddiioo__ffddbbuuff. + +IINNTTEERRNNAALLSS + When a ssuubbssttddiioo variable _s is used for input, there is + free buffer space from _s..xx to _s..xx + _s..nn; data is buffered + from _s..xx + _s..nn to _s..xx + _s..nn + _s..pp; the total buffer length + is _s..nn + _s..pp. + +SSEEEE AALLSSOO + substdio(3) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 2 + + + + + +substdio_out(3) substdio_out(3) + + +NNAAMMEE + substdio_out - substdio output routines + +SSYYNNTTAAXX + ##iinncclluuddee <> + + int ssuubbssttddiioo__ppuutt(&_s,_f_r_o_m,_l_e_n); + int ssuubbssttddiioo__ppuuttss(&_s,_f_r_o_m); + + int ssuubbssttddiioo__bbppuutt(&_s,_f_r_o_m,_l_e_n); + int ssuubbssttddiioo__bbppuuttss(&_s,_f_r_o_m); + + int ssuubbssttddiioo__fflluusshh(&_s); + + int ssuubbssttddiioo__ppuuttfflluusshh(&_s,_f_r_o_m,_l_e_n); + int ssuubbssttddiioo__ppuuttssfflluusshh(&_s,_f_r_o_m); + + substdio _s; + char *_f_r_o_m; + int _l_e_n; + +DDEESSCCRRIIPPTTIIOONN + ssuubbssttddiioo__ppuutt writes _l_e_n characters to _s out of the charac- + ter array _f_r_o_m. It returns 0 on success, -1 on error. + + ssuubbssttddiioo__bbppuutt has the same function as ssuubbssttddiioo__ppuutt. The + difference is how the buffer is flushed when there isn't + enough room for _l_e_n characters: ssuubbssttddiioo__ppuutt flushes the + buffered data before copying the new data, whereas ssuubbsstt-- + ddiioo__bbppuutt fills the buffer with new data before flushing. + + ssuubbssttddiioo__fflluusshh forces all data to be written from the + internal buffer. It returns 0 on success, -1 on error. + + ssuubbssttddiioo__ppuuttfflluusshh is similar to ssuubbssttddiioo__ppuutt followed by + ssuubbssttddiioo__fflluusshh, but it avoids all internal copies. + + ssuubbssttddiioo__ppuuttss, ssuubbssttddiioo__bbppuuttss, and ssuubbssttddiioo__ppuuttssfflluusshh are + the same as ssuubbssttddiioo__ppuutt, ssuubbssttddiioo__bbppuutt, and ssuubbssttddiioo__ppuutt-- + fflluusshh except that _f_r_o_m must be a 0-terminated string of + characters. The string, not including the 0, is written. + + The SSUUBBSSTTDDIIOO__OOUUTTSSIIZZEE macro is defined as a reasonably + large output buffer size for ssuubbssttddiioo__ffddbbuuff. + +IINNTTEERRNNAALLSS + When a ssuubbssttddiioo variable _s is used for output, data is + buffered from _s..xx to _s..xx + _s..pp; there is free buffer space + from to _s..xx + _s..pp to _s..xx + _s..nn; the total buffer length is + _s..nn. + +SSEEEE AALLSSOO + substdio(3) + + + + + 1 + + + + + +substdio_copy(3) substdio_copy(3) + + +NNAAMMEE + substdio_copy - copy an entire input to an output + +SSYYNNTTAAXX + ##iinncclluuddee <> + + int ssuubbssttddiioo__ccooppyy(&_s_o_u_t,&_s_i_n); + + substdio _s_o_u_t; + substdio _s_i_n; + +DDEESSCCRRIIPPTTIIOONN + ssuubbssttddiioo__ccooppyy reads characters from _s_i_n until end of file, + writing each character to _s_o_u_t. It then returns 0. It + does not flush _s_o_u_t. + + Upon a _s_i_n error, ssuubbssttddiioo__ccooppyy returns -2, leaving eerrrrnnoo + set appropriately. + + Upon a _s_o_u_t error, ssuubbssttddiioo__ccooppyy returns -3, leaving eerrrrnnoo + set appropriately. + +SSEEEE AALLSSOO + substdio(3) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 1 + +