# DIABLO.CONFIG - read by all diablo utilities to set diablo system-wide # options. Diablo does not auto-reparse this file on # modification, you must restart diablo. # # version X.Y # # options file version. The file must begin with a version command. # It then contains options related to that version. The file may then # optionally contain another version command and options related to THAT # version. Diablo will ignore options for versions greater then the # code scanning the options file. This allows diablo to report parsing # errors in a forward & backwards compatible manner. # version 2.0 ######################################################################### # # # PATHS # # # ######################################################################### # # NOTE: if 'active ' is specified later on, it overrides # any path_server_dactive specified here. # Root Directories (path_home based except for path_home itself) # #path_home /news #path_spool %s/spool/news #path_dqueue %s/dqueue #path_group %s/spool/group #path_cache %s/spool/cache #path_feeds %s/feeds #path_log %s/log #path_run %s/run # Configuration library and Database Roots # (path_home based) # #path_lib %s #path_db %s # Diablo and DReaderD (path_run based) # #path_diablo_socket %s/.diablosock #path_dreader_socket %s/.dreadersock #path_feednotify_socket %s/.feednotify # Diablo Configuration Paths (path_lib based) # #path_dexpire %s/dexpire.ctl #path_dcontrol %s/dcontrol.ctl #path_diablo_hosts %s/diablo.hosts #path_dserver_hosts %s/dserver.hosts #path_moderators %s/moderators #path_dnewsfeeds %s/dnewsfeeds #path_dnntpspool %s/dnntpspool.ctl #path_distrib_pats %s/distrib.pats #path_distributions %s/distributions #path_dspoolctl %s/dspoolctl #path_shutdown_cleanup NONE # Diablo Database Paths (path_db based) # #path_server_dactive %s/dactive.kp #path_reader_dactive %s/dactive.kp #path_dhistory %s/dhistory #path_spam_body_cache %s/spam.body.cache #path_spam_nph_cache %s/spam.nph.cache #path_pcommit_cache %s/pcommit.cache #path_dexpover_list %s/dexpover.dat #path_dhosts_cache %s/dhosts.cache #path_feedstats %s/feedstats #path_cachehits %s/cache.hits # Diablo Log Paths (path_log based) # # Logging paths can be specified as: # NONE no logging # SYSLOG log all data to syslog (except path_pathlog) # |/path/prog write log lines to a pipe # dir/path write log lines to the specified file # # path_generallog SYSLOG # (default) # path_incominglog %s/incoming.log # (default) # path_incominglog SYSLOG # path_incominglog NONE # (to disable) # path_drincominglog %s/drincoming.log # path_drincominglog SYSLOG # (default) # path_drincominglog NONE # (to disable - default) # path_pathlog %s/path.log # (log Path: lines) # path_pathlog |/dir/program # (write Path: to program) # path_pathlog NONE # (default) # # Log detailed article info into the specified file for stats gathering # Format is: # incoming_label size arttype newsgroups # path_artlog %s/article.log # (log host, groups, size) # path_artlog |/dir/program # (write artlog to program) # path_artlog NONE # (default) ######################################################################### # # # GENERAL OPTIONS # # # ######################################################################### # newsadmin news # The address used to display the admin user address for new # connections. # # newsadmin news@host.example.com # newsmaster usenet # The address to which mail about control messages is sent. Default # is "usenet". # # newsmaster usenet@host.example.com # displayadminversion on/off # Display the admin email address and Diablo version number # with connect and reject messages. The default is on. # #displayadminversion on ######################################################################### # # # DIABLO SERVER OPTIONS # # # ######################################################################### # hash METHOD[/COMPATIBLE-METHOD] # hash prime <--- original hash method # hash crc/prime <--- recommended when switching from prime # hash crc <--- recommended after stabilized with crc or w/ new # installation # # Specify the hash method. 'prime' is the original hash method. The new # hash method is 'crc'. The new crc method generates fewer collisions # and is recommended. # # You can have diablo fallback to an older hash method if a lookup using # the new method fails. This is only used when migrating from one hash # method to another and CAUSES DIABLO TO MAKE TWICE AS MANY HISTORY # LOOKUPS! The suggestion is that you set it to 'crc/prime' when # upgrading from diablo 1.13 or older. After a week or two of operation # you should then set it to just 'crc' and restart diablo. # # WARNING! changing the hash method will blow away the diablo reader # stuff and (usually) blow away the feeder's history file, so don't mess # with it unless you know what you are doing. # hash crc # hsize 1m/2m/4m/8m/16m ('k' or 'm' only) # # Hash table size, in entries. The default is 4 million entries (16MB). # The hash table is chained so you can't run out, but a smaller hash # table will result in more chain links and less efficient operation. # # This option can be changed on the fly, the next biweekly admin will # resize the hash table. Note that the -h option to diload will override # this setting just as this setting will override any compiled default. hsize 4m # active on/off # activedrop on/off # # Turn on or off dactive.kp based Xref generation in the diablo feeder. # The default is off. Turning this option on will cause diablo to # assign article numbers for those groups found in dactive.kp. The NX # field (alternate ending-article-number) in dactive.kp will be updated # appropriately. If slave mode is off, Xref: headers will be generated # based on the updated NX field for a group in dactive.kp. # # activedrop defaults to off. The diablo feeder normally spools # articles even if none of the article's groups can be found in the # active file. If activedrop is set to "on", the diablo feeder will # drop such articles. # # If you specify an absolute path instead of "on", you turn on active # and override path_server_dactive. This feature is depreciated in # favor of the path_server_dactive mechanism and should no longer be # used. # # Please see the manual page for 'diablo-files', 'dkp' and 'diablo-kp' # for reference. # # -- # # If you use the active file feature on the feeder side it is usually # in order to generate master article reference numbers for one or more # readers via Xref:. In this case you typically want to leave the # server-side slave mode off (slave off), and you typically want to # start the diablo reader side (dreaderd) with the '-x masterhostname' # option. # # NOTE! dreaderd is responsible for updating the NB and NE fields # for a group in dactive.kp, where NE is the primary ending article # number in the group. When acting as a master the Diablo server # will update the NX field, allowing it to operate on the same dactive.kp # file as a dreaderd running on the same host. It is not appropriate # for the diablo server to update NE because that would result in # incorrect output from the 'group' and 'list active' commands, possibly # reporting the existance of articles not yet placed in overview. This # is why the diablo server updates NX. When synchronizing your active # file to a remote host with the dsyncgroups commands, pay special # attention to the -X and -XX options to dsyncgroups. # active off activedrop off # feederxrefslave on/off # slave on/off (This is the old option now replaced with feederxrefslave) # # Turn on or off Xref: slave mode. The default is off, which causes # Diablo to strip incoming Xref: headers and, if active is on, # generate it's own Xref: headers. # # If slave mode is on Diablo will not strip incoming Xref: headers and # will NOT generate it's own. # feederxrefslave off # feederxrefsync on/off # # If feederxrefslave is on and active is on, then sync our active NX # field (end article number) for each newsgroup in the Xref: line if # one exists in the article. # # feederxrefsync off # feederxrefhost hostname # # Specify the Xref: hostname to be used when creating Xref: lines # This defaults to the pathhost # # feederxrefhost host.example.com # maxconnect N # maxconnect 15 # # Specify the maximum number of parallel connections per connecting IP. # This is an overall maximum. The maxconnect field in dnewsfeeds must # be less then this parameter to override it. # # IF DREADERD CONNECTS TO THIS DIABLO, USE A LARGE MAXCONNECT (like 200). # DREADERD MAKES A CONNECTION TO EACH SERVER FOR EVERY FORK!!! # Additionally, if you have configured dreaderd to use the diablo server # for both posting and spool access, the number of connections will # double. Be careful in regards to running the diablo server out of # file descriptors! # maxconnect 15 # remember N # # Specify the history retention for diablo. If not specified here, # the compiled default (14) is used. This parameter will override the # compiled default for both the diablo binary and the didump binary's # -x option. # # Note that history entries for articles still in the spool (as # determined by dexpire) are not deleted even if they are older then # the remember time. # # This parameter also specifies the oldest article that Diablo # will accept on an incoming newsfeed. See also feedermaxacceptage. # # Caution: Diablo does not examine the time zone offset for an # article when deciding if that article is too old or too far into # the future to be accepted. Because of this, setting "remember" # to 1 may cause cause some articles posted with non local time zone # indicators to be rejected. As this side-effect is not that # obvious, it might be better to consider the "remember" setting as # establishing the absolute maximum article age that will be # accepted, with the minumum age that could be accepted being 24 # hours less, minus your local offset to GMT. # #remember 15 #remember 7 #remember 3 #feedermaxacceptage N # Specify the maximum age for received articles before they are # rejected (in days). A 's', 'm', 'h' or 'd' can be appended to # the value to indicate seconds, minutes, hours or days respectively. # The default is 14 days. # # Caution: Diablo does not examine the time zone offset for an # article when deciding if that article is too old or too far into # the future to be accepted. Because of this, setting this value # to 1 may cause cause some articles posted with non local time zone # indicators to be rejected. As this side-effect is not that # obvious, it might be better to consider the "remember" setting as # establishing the absolute maximum article age that will be # accepted, with the minumum age that could be accepted being 24 # hours less, minus your local offset to GMT. # # feedermaxacceptage 10h # feedermaxacceptage 14 # hostcacherebuildtime n # Specify the time (in seconds) between rebuilds of the diablo.hosts # host IP address cache. This cache file is used to improve the # performance of new connections and to minimise DNS lookups. # The default of every hour is probably more than sufficient # to catch changes to the DNS. The file is also rebuilt within a # minute of diablo.hosts being modified. # #hostcacherebuildtime 3600 # precommittime n # Set the amount of time (in seconds) that entries are considered # valid in the precommit cache (cache of message-id's currently # being offered to us). The default is 30 seconds. A value of # 0 disables the cache and negative values are ignored. #precommittime 30 # postcommittime n # Set the amount of time (in seconds) that entries are considered # valid in the postcommit cache (history cache). The default is # 45 minutes (2700 sec). A value of 0 disables the cache and # negative values are ignored. Note that cache slots can be # reused within this time, so it is only a maximum value. #postcommittime 2700 # hiscachesize n # Set the number of slots available in the history cache. The # actual memory used is this value x the size of the history # cache structure (currrently 16 bytes). # IMPORTANT: This value needs to be a power of 2. e.g: 8192, # 16384, 32768, 65536, 131072 #hiscachesize 16384 # feederhostname hostname # Set the hostname of this host. This value is used to set the # defaults for other host options (pathhost, xref, etc) and # defaults to the value obtained by the gethostname() call. # # feederhostname host.example.com # feederbindport port # Set the TCP port to listen for connections. Defaults to the # NNTP port of 119. feederbindport 119 # feederbindhost hostname/ip # Only listen on a specific interface specified as a hostname or # IP address. Default is all interfaces. # feederbindhost # internalfilter n # Enable the internal spamfilter. IMPORTANT: There must also be # an ISPAM entry in dnewsfeeds before any articles will reach the # filter even if it is enabled here. The ISPAM entry determines # which articles are sent to the internal filter. # # There are two internal filters available: # Duplicate body detection # NNTP-Posting-Host rate detection # Each type is enabled with a different option, which also sets # the trip value for that type. # Bnnn - enables duplicate body detection and sets the number # of allowed duplicates before further articles are rejected. # Nnnn - enables the NNTP-Posting-Host rate detection and the # number specifies how many duplicate hosts are allowed # in an hour before extra articles from that host are # rejected. # ennn - set the expire time (in seconds) for the previous # 'B' or 'N' option. Default is 3600 (1 hour). # snnn - set the number of entries in the filter hash table to # nnn for the previous 'B' or 'N' option. The size must be # a power of 2. Default is 65536 entries. # # Both types of filters also make a note of the number of lines in # the body of the article to reduce the possibility of false # duplicates. # # Use of this option causes the creation of 2 files in path_db # that are used to store the filter hash tables. # # e.g: B6s32768 N16t7200 would set the body filter trip to 6, with # a hash table size of 32768 entries and the nph filter trip to 16 # with the default hash table size of 65536 and keeps track of # entries for 2 hours. # # The default is disabled (B0 N0). # # internalfilter B6 N16 # feederfilter /filterpath # Set the path to and enable the external spamfilter. # Note that there must also be an ESPAM entry in dnewsfeeds # which determines which articles are sent to the filter and # the dnewsfeed labels for feeds will need 'nospam' directives # to prevent the feed from receiving spam. # By default, this option is disabled and no articles are sent # to the external spamfilter. # # feederfilter /news/dbin/filter/spamfilter # rejectartswithnul on/off # Enable/disable the rejection of articles that contain a NUL # character. Default is off. # rejectartswithnul off # feederpath hostname # Set the pathhost value added to all articles accepted by the # feeder. The pathhost is added when the article is received. # Defaults to the feederhostname. A value of '0' will disable # the addition of the hostname to the Path: header, but this # use is discouraged as it can cause loops if not used carefully. # Multiple uses of this option can be used and will be added # to the header in the order in which they appear in this file. # See also the commonpath option, which is included in the oder # calculation. # # feederpath host.example.com # commonpath # Add this to the Path: header if it doesn't already exist there. # Multiple uses of this option can be used and will be added # to the header in the order in which they appear in this file. # See also the feederpath option, which is included in the oder # calculation. This defaults to blank. # # commonpath host.example.com # feederautoaddtoactive on/off # Automatically add all newsgroups in the Newsgroup: header that don't # exist in the feeder's active file. Default is off. # # Note that it won't get a group description (GD), moderator address # (M), or moderation flag (S) set, which is almost certainly # something you'll have to cobble together yourself at a later date. # # You probably do NOT want to set this on any normal newsserver # # feederautoaddtoactive off # feederrtstats none/label/hostname # Keep realtime statistics of incoming and outgoing feeds by # label or hostname. The default is by label and 'none' disabled # the collection of realtime feed stats. # # The stats are stored in a file specified by path_feedstats # and are retrieved using dfeedinfo. The file is only created # when the first connection is made (incoming or outgoing). The # stats are persisent across reboots and only ever reset when # when done manually by deleting the file or using dfeedinfo -z. # # feederrtstats label # feederbuffersize value # Set the starting size of the incoming article buffer. This # buffer is allocated each time an article comes in and increasing # this has slight benefits when receiving large binaries. # The higher this value, the more memory the diablo processes use. # The default value is 1, which means use the pagesize for the # OS (generally 4096 bytes). The pagesize is the minimum value. # A value of '32k' may be useful. # # feederbuffersize 1 # feedermaxartsize n # Set the maximum incoming article size accepted by diablo. # This is a good way to restrict the size of buffers used for # incoming articles. Without this option, a very large article # could use a lot of memory. This option doesn't save bandwidth # as the article is still received; it just isn't stored anywhere. # The default is 1000000. A value of 0 allows any article size. # feedermaxartsize 10000000 # wireformat on/off # This option causes all articles to be stored on the spool in # the format they are received from the remote host. i.e: They # include the '\r' and initial line '.'s are not decoded. The # article is also stored with the trailing '.\r\n'. This causes # a little more disk space to be used to gain performance in # sending articles out (CPU saving). # # wireformat off # feederarttypes on/off # This option allows the article type detection code to be # disabled. This can have a slight performance improvement on # feeders that don't use any form of article type decisions. # Default: on # # feederarttypes on # feederpreloadart on/off # spoolpreloadart on/off # These options enable/disable an option to the loading of # articles via mmap() that cause the articles to be preloaded # thus potentially decreasing the number of seeks and disk # activity at the expense of memory. Not all operating systems # support the MADV_WILLNEED option to madvise() used by this # option and hence it will be disabled on them. feederpreloadart # applies to dnewslink and spoolpreloadart applies to retrieval # of articles from a diablo spool. # Default: on (if available) # # feederpreloadart on # spoolpreloadart on ######################################################################### # # # DIABLO READER OPTIONS # # # ######################################################################### # readerforks 10 # readerthreads 40 # readerdns 5 # readerfeedforks 2 # # Specify the number of reader processes (forks), and number of # threads per process. A thread limitation exists dependant on # the number of descriptors your system supports, one descriptor # is required per thread. Also specify the number of DNS reverser # processes. # # A fork limitation exists in that each reader fork will open up all # server connections (/news/dserver.hosts), eating a base number of # descriptors for fork. # # The default is 10 forks, 40 threads = 400 online readers. You can # up these values considerably. for example, 25 forks and 40 threads # would support 1000 simultanious readers. # # WARNING! Each fork connects to all spool/reader servers, so make sure # they can handle it. For example, if connecting to a diablo spool or # feed you must set 'maxconnect' to at least the number of readerforks!!! # It may be better to do 10 forks and 40 threads, for example. Watch out # for your file descriptor usage... I do not suggest more then 64 threads # per fork. If you experience excessive latency, try increasing the # number of forks and decreasing the number of threads per fork. # # The readerfeedfork option specified the number of dedicated # feed-only processes are created. Feed-only processes have a slight # performance improvement for incoming header feeds over connections # shared with reader processes. # readercache on # # Specify whether dreaderd tries to cache articles in /news/spool/cache/ # or not. The default is on. # # You must periodically remove files from the cache from cron. You can # remove any file at any time. Please see the sample cron jobs. YOU # ARE RESPONSIBLE FOR MAINTAINING FREE SPACE IN THE CACHE DIRECTORY. # Dreaderd will not die if it runs out of space in the cache but it will # become more inefficient. # # You should turn off the readercache if you are running the diablo feeder # on the same machine. Use dserver.hosts to make the feeder the first # level cache for the reader. # readercachehashsize 2048 # # set the number of entries for the hash of the scoring cache. # default to 4096 # readerxover on/trackonly # # Default is on. Specify how the reader is to maintain its xover # information. If "on" the reader maintains the article numbering # control information used by expireover to maintain the active file # AND the reader stores the article headers. This is required for # normal reader operation where a user connects in with a newsreader # client. # # If you are using the reader *just* so you can maintain the active # file's article numbering and run expireover then you can set this # to 'trackonly' which causes the control information to be stored but # the headers not to be stored. 'trackonly' mode is often used when # you want to setup one machine to master and maintain the spool and # active file, and have other machines deal with the user readers. # readercrash none # readercrash /news/bin/dreaderd-crash-handler # # This specifies the program to run if dreaderd crashes. In case of # a SIGSEGV, SIGBUS, etc, dreaderd will fork, and run this program, # passing the forked PID as the only argument. Since dreaderd cannot # dump core, this is an alternative way of preserving crash state; # if the handler notifies the maintainers, they can later attach to # the crashed process and examine what happened. # # Default is "none", which preserves the old behavior of chewing all # spare CPU cycles at high niceness, if dreaderd is ran with -X. If # a value other than none is specified, -X is not necessary. # readerhostname hostname # Set the hostname of this host. This value is used to set the # defaults for other host options (pathhost, xref, etc) and # defaults to the value obtained by the gethostname() call. # # readerhostname host.example.com # readerbindport port # Set the TCP port to listen for connections. Defaults to the # NNTP port of 119. readerbindport 119 # readerbindhost hostname/ip # Only listen on a specific interface specified as a hostname or # IP address. Default is all interfaces. # readerbindhost # readerxrefhost hostname # Specify the Xref: hostname to be used when displaying Xref: lines # This defaults to the pathhost and can be overrriden we per # connection using options in dreader.access # # readerxrefhost host.example.com # readerxrefslavehost hostname # Specify the Xref: hostname that must match incoming articles # before they are accepted into the header database. Articles # with an Xref: hostname that doesn't match this option will # be dropped. This option must be specified if the reader is # running in Xref slave mode. If it and the '-x' command-line # option are not specified, dreaderd will ignore Xref: lines # of incoming articles and generate its own. # # readerxrefslavehost host.example.com # readerpath hostname # Set the value added as the first (most recent) entry in the # Path: header when sending articles to clients. # This defaults to the pathhost and can be overrriden we per # connection using options in dreader.access # # readerpath host.example.com # readerautoaddtoactive on/off # Automatically add all newsgroups in the Newsgroup: header that don't # exist in the reader's active file. This includes local postings. # Default is off. # # Note that it won't get a group description (GD), moderator address # (M), or moderation flag (S) set, which is almost certainly # something you'll have to cobble together yourself at a later date. # # You probably do NOT want to set this on any normal newsserver # # readerautoaddtoactive off # readerdetaillog on/off # Log detailed debugging information about incoming and outgoing # commands via syslog. Default is on. # # This can also be configured in dreader.access with the "logcmd" # command on a per-reader basis. # # readerdetaillog on # readercachedirs N[/N]... # Set the number of reader cache directory levels and directories # per level to create. Up to 9 levels of N directories can be # specified. Directories are created automatically if they don't # exist and are made from a hex value of the hash of the cache # file name. At least 2 levels of directories is a good option. # The default (backward compatable) is a single level of 512 # directories. # IMPORTANT: If this option is changed, the cache spool MUST be # deleted to prevent incorrect articles being sent to clients. # readercachedirs 512 # readergrouphash crc (default) # readergrouphash md5-32[:B]/N[/N] # readergrouphash md5-32[:B]\N[\N] # readergrouphash md5-64[:B]/N[/N] # readergrouphash md5-64[:B]\N[\N] # readergrouphash hierarchy # Set the hash method used for newsgroup header index and data # filenames. # crc is the old (and default) method, but has been found to have # duplicate hashes for groups with very similar names. # md5-32 is a 32 bit base64 conversion of an md5 hash of the group name. # md5-64 is a base64 type conversion of an md5 hash of the group name, # The backslash ('\') followed by the number of characters represents # the number of characters of the hash (from the start) to use for the # directory name (max 9). A 2nd level of directories is optional # and the characters after the 1st level are used. # The forwardslash ('/') followed by a number represents the number # of directories to create at that level. # See lib/hash.c for a more comprehensive description of the # directories and structure. # hierarchy uses the newsgroup hierarchy for the path and the full # newsgroup name for the filename. This option is not recommended # for large servers as it can lead to very unbalanced directory # sizes and hence performance problems. # # Once a method is chosen you can only switch to another with the # server down and by running 'doverctl'. Also ensure that dexpireover # does not run during the conversion. # # md5-32/128/128 is recommended for an average full feed server on # an array of disks. This seems to give an average spread of groups # over 16384 directories. # # readergrouphash crc # # readerban UsNNlNNcNNtNN # readerban HsNNlNNcNNtNN # readerban GsNNlNNcNNtNN # readerban LsNNlNNcNNtNN # Enable the use of a ban list for abusive reader connections. # NOTE: This option needs to be enabled at compile-time (lib/vendor.h) # This option works by counting the connections that exceed the # limits for the specified category of connection and once exceeded # further connections from that user/host are rejected until the # specified timeout. Any connections attempts during the banned # period, reset the timer from the start. # The options are: # U following options are for authenticated users # (maxconnperuser) # H following options are for hosts (maxconnperhost) # G following options are for groups (maxconnpergroup) # L following options are global (connections exceeding # the server limits. # s set the hash size (default is 64 entries) # l set the length of the hash links (default 8) # c set the number of connection attempts before # the user/host is banned # t set the amount of time (in seconds) for which # the user/host is banned # Spaces are not permitted. The lower case letters apply to # the type of connections specified by the capital letter prior # to it in the list. e.g: # Hc10t3600Uc10t7200 # means that any host that exceeds the maxconnperhost more than 10 # times, will have connections rejected for the next hour. Users # will be rejected for the next 2 hours. # This option is not really suitable for general news server use, # but can be useful for servers that are often abused by users # trying to exceed the access configured limits. # The default is no banning. # # readerban # readermaxartsize n # Set the maximum incoming article size accepted by dreaderd # via a local POST or incoming header feed. # This is a good way to restrict the size of buffers used for # incoming articles. Without this option, a very large article # could use a lot of memory. This option doesn't save bandwidth # as the article is still received; it just isn't stored anywhere. # The default is 1000000. A value of 0 allows any article size. # readermaxartsize 10000000 # pgpverifyargs string # Set the arguments to pgpverify. This is usually the path and # arguments used to run pgp or gpg # PGP 2.6.1i: /usr/local/bin/pgp -f # GPG : /usr/local/bin/gpgv # pgpverifyargs /usr/local/bin/pgp -f