<HTML>
<HEAD>
<META NAME="GENERATOR" CONTENT="SGML-Tools 1.0.9">
- <TITLE>The DXSpider Installation and Administration Manual : Configuration</TITLE>
+ <TITLE>The DXSpider Administration Manual v1.47: Filtering (New Style v1.45 and later)</TITLE>
<LINK HREF="adminmanual-4.html" REL=next>
<LINK HREF="adminmanual-2.html" REL=previous>
<LINK HREF="adminmanual.html#toc3" REL=contents>
+<link rel=stylesheet href="style.css" type="text/css" title="default stylesheet">
</HEAD>
<BODY>
<A HREF="adminmanual-4.html">Next</A>
<A HREF="adminmanual-2.html">Previous</A>
<A HREF="adminmanual.html#toc3">Contents</A>
<HR>
-<H2><A NAME="s3">3. Configuration</A></H2>
+<H2><A NAME="s3">3. Filtering (New Style v1.45 and later)</A></H2>
-<H2><A NAME="ss3.1">3.1 Allowing ax25 connects from users</A>
+<H2><A NAME="ss3.1">3.1 General filter rules</A>
</H2>
-<P>As stated previously, the aim of this document is not to tell you how to configure Linux or the ax25 utilities. However,
-you do need to add a line in your ax25d.conf to allow connections to DXSpider for your users. For each interface that
-you wish to allow connections on, use the following format ...
+<P>Upto v1.44 it was not possible for the user to set their own filters. From
+v1.45 though that has all changed. It is now possible to set filters for just
+about anything you wish. If you have just updated from an older version of
+DXSpider you will need to update your new filters. You do not need to do
+anything with your old filters, they will be renamed as you update.
<P>
-<BLOCKQUOTE><CODE>
-<PRE>
-default * * * * * * - sysop /spider/src/client client %u ax25
-</PRE>
-</CODE></BLOCKQUOTE>
+<P>There are 3 basic commands involved in setting and manipulating filters. These
+are <EM>accept</EM>, <EM>reject</EM> and <EM>clear</EM>. First we will look
+generally at filtering. There are a number of things you can filter in the
+DXSpider system. They all use the same general mechanism.
<P>
-<H2><A NAME="ss3.2">3.2 Allowing telnet connects from users</A>
-</H2>
-
-<P>Allowing telnet connections is quite simple. Firstly you need to add a line in /etc/services to allow connections to a
-port number, like this ....
+<P>In general terms you can create a 'reject' or an 'accept' filter which can have
+up to 10 lines in it. You do this using, for example ...
<P>
<BLOCKQUOTE><CODE>
<PRE>
-spdlogin 8000/tcp # spider anonymous login port
+
+accept/spots .....
+reject/spots .....
</PRE>
</CODE></BLOCKQUOTE>
-<P>Then add a line in /etc/inetd.conf like this ....
+<P>where ..... are the specific commands for that type of filter. There are filters
+for spots, wwv, announce, wcy and (for sysops) connects. See each different
+accept or reject command reference for more details.
+<P>There is also a command to clear out one or more lines in a filter. They are ...
<P>
<BLOCKQUOTE><CODE>
<PRE>
-spdlogin stream tcp nowait root /usr/sbin/tcpd /spider/src/client login telnet
+clear/spots 1
+clear/spots all
</PRE>
</CODE></BLOCKQUOTE>
+<P>There is clear/xxxx command for each type of filter.
<P>
-<P>This needs to be added above the standard services such as ftp, telnet etc. Once this is done, you need to restart inetd
-like this ....
+<P>and you can check that your filters have worked by the command ...
<P>
<BLOCKQUOTE><CODE>
<PRE>
-killall -HUP inetd
+
+show/filter
</PRE>
</CODE></BLOCKQUOTE>
<P>
+<P>For now we are going to use spots for the examples, but you can apply the same
+principles to all types of filter.
<P>
-<P>Now login as <EM>sysop</EM> and cd spider/perl. You can test that spider is accepting telnet logins by issuing the
-following command ....
-<P>
-<BLOCKQUOTE><CODE>
-<PRE>
-client.pl login telnet
-</PRE>
-</CODE></BLOCKQUOTE>
-<P>You should get a login prompt and on issuing a callsign, you will be given access to the cluster. Note, you will not
-get a password login. There seems no good reason for a password prompt to be given so it is not asked for.
-<P>
-<P>Assuming all is well, then try a telnet from your linux console ....
-<P>
-<BLOCKQUOTE><CODE>
-<PRE>
-telnet localhost 8000
-</PRE>
-</CODE></BLOCKQUOTE>
-<P>
-<P>You should now get the login prompt and be able to login as before.
-<P>
-<H2><A NAME="ss3.3">3.3 Setting up node connects</A>
+<H2><A NAME="ss3.2">3.2 Types of filter</A>
</H2>
-<P>In order to allow cluster node connections, spider needs to know that the connecting callsign is a cluster node. This
-is the case whether the connect is incoming or outgoing.
-In spider this is a simple task and can be done in runtime.
+<P>There are two main types of filter, <EM>accept</EM> or <EM>reject</EM>. You
+can use either to achieve the result you want dependent on your own preference
+and which is more simple to do. It is pointless writing 8 lines of reject
+filters when 1 accept filter would do the same thing! Each filter has 10
+lines (of any length) which are tried in order. If a line matches then the
+action you have specified is taken (ie reject means ignore it and accept
+means take it)
<P>
-<P>Start up the cluster as you did before and login as the sysop with client.pl.
-The cluster node I am wanting to make a connection to is GB7BAA but you would obviously use whatever callsign you
-required.
-At the prompt type ...
+<P>If you specify reject filters, then any lines that arrive that match the filter
+will be dumped but all else will be accepted. If you use an accept filter,
+then ONLY the lines in the filter will be accepted and all else will be dumped.
+For example if you have a single line <EM>accept</EM> filter ...
<P>
<BLOCKQUOTE><CODE>
<PRE>
-set/node gb7baa
+accept/spots on vhf and (by_zone 14,15,16 or call_zone 14,15,16)
</PRE>
</CODE></BLOCKQUOTE>
+<P>then you will <EM>ONLY</EM> get VHF spots <EM>from</EM> or <EM>to</EM> CQ zones
+14, 15 and 16.
<P>
-<P>The case does not matter as long as you have a version of DXSpider later than 1.33. Earlier versions required the
-callsign to be in upper case.
-<P>
-<P>That is now set, it is as simple as that. To prove it, login on yet another console as sysop and issue the command ...
+<P>If you set a reject filter like this ...
<P>
<BLOCKQUOTE><CODE>
<PRE>
-client.pl gb7baa (using the callsign you set as a node)
+reject/spots on hf/cw
</PRE>
</CODE></BLOCKQUOTE>
-<P>
-<P>You should get an initialisation string from DXSpider like this ...
+<P>Then you will get everything <EM>EXCEPT</EM> HF CW spots. You could make this
+single filter even more flexible. For example, if you are interested in IOTA
+and will work it even on CW even though normally you are not interested in
+CW, then you could say ...
<P>
<BLOCKQUOTE><CODE>
<PRE>
-client.pl gb7baa
-PC38^GB7MBC^~
+reject/spots on hf/cw and not info iota
</PRE>
</CODE></BLOCKQUOTE>
-<P>If the callsign you just set up as a cluster node is for an incoming connect, this is all that needs to be done.
-If the connection is to be outgoing then a connection script needs to be written.
-<P>
-<H2><A NAME="ss3.4">3.4 Connection scripts</A>
-</H2>
-
-<P>Because DXSpider operates under Linux, connections can be made using just about any protocol; AX25, NETRom, tcp/ip,
-ROSE etc are all possible examples. Connect scripts live in the /spider/connect directory and are simple ascii files.
-Writing a script for connections is therefore relatively simple.
-<P>
-<P>The connect scripts consist of lines which start with the following keywords or symbols:-
-<P>
-<P>
-<PRE>
-
-# All lines starting with a # are ignored, as are completely blank lines.
-
-timeout timeout followed by a number is the number of seconds to wait for a
- command to complete. If there is no timeout specified in the script
- then the default is 60 seconds.
-
-abort abort is a regular expression containing one or more strings to look
- for to abort a connection. This is a perl regular expression and is
- executed ignoring case.
-
-connect connect followed by ax25 or telnet and some type dependent
- information. In the case of a telnet connection, there can be up to
- two parameters.
- The first is the ip address or hostname of the computer you wish to
- connect to and the second is the port number you want to use (this
- can be left out if it is a normal telnet session).
- In the case of an ax25 session then this would normally be a call to
- ax25_call or netrom_call as in the example above. It is your
- responsibility to get your node and other ax25 parameters to work
- before going down this route!
-
-' ' is the delimiting character for a word or phrase of an expect/send
- line in a chat type script. The words/phrases normally come in pairs,
- either can be empty. Each line reads input from the connection until
- it sees the string (or perl regular expression) contained in the
- left hand string. If the left hand string is empty then it doesn't
- read or wait for anything. The comparison is done ignoring case.
- When the left hand string has found what it is looking for (if it is)
- then the right hand string is sent to the connection.
- This process is repeated for every line of chat script.
-
-client client starts the connection, put the arguments you would want here
- if you were starting the client program manually. You only need this
- if the script has a different name to the callsign you are trying to
- connect to (i.e. you have a script called other which actually
- connects to GB7DJK-1 [instead of a script called gb7djk-1]).
-</PRE>
-<P>
-<P>There are many possible ways to configure the script but here are two examples, one for a NETRom/AX25 connect and
-one for tcp/ip.
+<P>But in that case you might only be interested in iota and say:-
<P>
<BLOCKQUOTE><CODE>
<PRE>
-timeout 60
-abort (Busy|Sorry|Fail)
-# don't forget to chmod 4775 netrom_call!
-connect ax25 /usr/sbin/netrom_call bbs gb7djk g1tlh
-'Connect' ''
-'Connect' 'c np7'
-'Connect' 'c gb7dxm'
-# you can leave this out if you call the script 'gb7dxm'
-client gb7dxm ax25
+accept/spots not on hf/cw or info iota
</PRE>
</CODE></BLOCKQUOTE>
+<P>which achieves exactly the same thing. You should choose one or the other
+until you are comfortable with the way it works. You can mix them if you
+wish (actually you can have an accept AND a reject on the same line) but
+don't attempt this until you are sure you know what you are doing!
<P>
-<P>
+<P>You can arrange your filter lines into logical units, either for your own
+understanding or simply convenience. Here is an example ...
<P>
<BLOCKQUOTE><CODE>
<PRE>
-timeout 15
-connect telnet dirkl.tobit.co.uk
-'login' 'gb7djk'
-'word' 'gb7djk'
-# tell GB7DJK-1 that it is connected to GB7DJK
-# you can leave this out if you call this script 'gb7djk'
-client gb7djk telnet
+reject/spots 1 on hf/cw
+reject/spots 2 on 50000/1400000 not (by_zone 14,15,16 or call_zone 14,15,16)
</PRE>
</CODE></BLOCKQUOTE>
+<P>What this does is to ignore all HF CW spots and also rejects any spots on VHF
+which don't either originate or spot someone in Europe.
<P>
-<P>Both these examples assume that everything is set up properly at the other end. You will find other examples in
-the /spider/examples directory.
+<P>This is an example where you would use a line number (1 and 2 in this case), if
+you leave the digit out, the system assumes '1'. Digits '0'-'9' are available.
+This make it easier to see just what filters you have set. It also makes it
+more simple to remove individual filters, during a contest for example.
<P>
-<H2><A NAME="ss3.5">3.5 Starting the connection</A>
-</H2>
-
-<P>You start the connection, from within a sysop enabled cluster login, by typing in the word <EM>connect</EM> followed
-by a script name like this ....
+<P>You will notice in the above example that the second line has brackets. Look
+at the line logically. You can see there are 2 separate sections to it. We
+are saying reject spots that are VHF or above <EM>APART</EM> from those in
+zones 14, 15 and 16 (either spotted there or originated there). If you did
+not have the brackets to separate the 2 sections, then Spider would read it
+logically from the front and see a different expression entirely ...
<P>
<BLOCKQUOTE><CODE>
<PRE>
-G0VGS de GB7MBC 13-Dec-1998 2041Z >connect gb7djk-1
-connection to GB7DJK-1 started
-G0VGS de GB7MBC 13-Dec-1998 2043Z >
+(on 50000/1400000 and by_zone 14,15,16) or call_zone 14,15,16
</PRE>
</CODE></BLOCKQUOTE>
-<P>This will start a connection using the script called <EM>gb7djk-1</EM>. You can follow the connection by watching the
-term or console from where you started <EM>cluster.pl</EM>. You should see something like this ...
+<P>The simple way to remember this is, if you use OR - use brackets. Whilst we are
+here CASE is not important. 'And BY_Zone' is just the same as 'and by_zone'.
+<P>As mentioned earlier, setting several filters can be more flexible than
+simply setting one complex one. Doing it in this way means that if you want
+to alter your filter you can just redefine or remove one or more lines of it or
+one line. For example ...
<P>
<BLOCKQUOTE><CODE>
<PRE>
-<- D G1TLH connect gb7djk-1
--> D G1TLH connection to GB7DJK-1 started
--> D G1TLH G1TLH de GB7DJK 13-Dec-1998 2046Z >
-timeout set to 15
-CONNECT sort: telnet command: dirkl.tobit.co.uk
-CHAT "login" -> "gb7djk"
-received "
-Red Hat Linux release 5.1 (Manhattan)
-Kernel 2.0.35 on an i586
-"
-received "login: "
-sent "gb7djk"
-CHAT "word" -> "gb7djk"
-received "gb7djk"
-received "Password: "
-sent "gb7djk"
-Connected to GB7DJK-1, starting normal protocol
-<- O GB7DJK-1 telnet
--> B GB7DJK-1 0
-GB7DJK-1 channel func state 0 -> init
-<- D GB7DJK-1
-<- D GB7DJK-1 Last login: Sun Dec 13 17:59:56 from dirk1
-<- D GB7DJK-1 PC38^GB7DJK-1^~
-<- D GB7DJK-1 PC18^ 1 nodes, 0 local / 1 total users Max users 0 Uptime 0 00:00^5447^~
- etc
+reject/spots 1 on hf/ssb
</PRE>
</CODE></BLOCKQUOTE>
-<P>
-<P>With later versions of Spider there is a set/login command for users. This tells them when a user or node logs in
-or out. If you do not add a line to your scripts after the final line (or before the client line which should always
-be last if needed) then the login/logout information will be sent to users <I>before</I> the login actually
-completes. This means if a node is unreachable, it will continue sending logins and logouts to users even though it
-is not actually connecting. To avoid this use the following line ...
+<P>would redefine our earlier example, or
<P>
<BLOCKQUOTE><CODE>
<PRE>
-'connect' ''
+clear/spots 1
</PRE>
</CODE></BLOCKQUOTE>
-<P>
-<P>In a script, this might look like ...
+<P>To remove all the filter lines in the spot filter ...
<P>
<BLOCKQUOTE><CODE>
<PRE>
-timeout 35
-abort (Busy|Sorry|Fail)
-connect telnet mary 3000
-'ogin:' 'gb7mbc'
-'>' 'telnet 44.131.93.96 7305'
-'connect' ''
+clear/spots all
</PRE>
</CODE></BLOCKQUOTE>
<P>
-<H2><A NAME="ss3.6">3.6 Telnet echo</A>
+<H2><A NAME="ss3.3">3.3 Filter options</A>
</H2>
-<P>Cluster links in particular suffer greatly from the presence of telnet echo. This is caused
-by the telnet negotiation itself and can create at worst severe loops. At best it creates
-unnecessary bandwidth and large logfiles! There are things that can be done to limit this
-problem but will not always work dependent on the route taken to connect.
-<P>
-<P>Telnet echo itself should only be a problem if the connection is being made to the telnet
-port (23). This port uses special rules that include echo negotiation. If the connection
-is to a different port, such as 8000, this negotiation does not happen and therefore no
-echo should be present.
-<P>
-<P>Sometimes it is not possible to make a direct connection to another node and this can
-cause problems. There is a way of trying to suppress the telnet echo but this will not
-always work, unfortunately it is difficult to be more specific. Here is an example
-of what I mean ...
-<P>
-<BLOCKQUOTE><CODE>
-<PRE>
-timeout 35
-abort (Busy|Sorry|Fail)
-connect telnet mary.lancs.ac.uk
-'ogin:' 'gb7mbc'
-'word:' 'mypasswd'
-'\$' 'stty -echo raw'
-'\$' 'telnet 44.131.93.96'
-'connect' ''
-</PRE>
-</CODE></BLOCKQUOTE>
-<P>So, the first connection is made by Spider. This is fine as Spider uses the Net_Telnet
-script from within perl. This actually uses TCP rather than TELNET so no negotiation
-will be done on the first connection. Once connected to mary.lancs.ac.uk, the command
-is sent to suppress echo. Now a telnet is made to a cluster node that is accepting
-connections on port 23. The problem with this link is that the negotiation is made by
-the remote machine, therefore you have no control over it. The chances are that this
-link will create echo and there will be no way you can stop it.
+<P>You can filter in several different ways. The options are listed in the
+various helpfiles for accept, reject and filter.
<P>
-<P>
-<H2><A NAME="ss3.7">3.7 Automating things</A>
+<H2><A NAME="ss3.4">3.4 Default filters</A>
</H2>
-<P>Ok, you should now have DXSpider running nicely and allowing connects by cluster nodes or users. However, it has
-to be shutdown and restarted manually and if connection scripts fail they have to be started again manually too, not
-much use if you are not at the console! So, in this section we will automate both. Firstly starting the cluster.
-<P>
-<H3>Autostarting the cluster</H3>
-
-<P>This is not only a way to start the cluster automatically, it also works as a watchdog, checking the sanity of
-DXSpider and respawning it should it crash for any reason.
-Before doing the following, shutdown the cluster as you did earlier.
-<P>
-<P>Login as root and bring up the /etc/inittab file in your favourite editor. Add the following lines to the file near
-the end ...
+<P>Sometimes all that is needed is a general rule for node connects. This can
+be done with a node_default filter. This rule will always be followed, even
+if the link is isolated, unless another filter is set specifically. Default
+rules can be set for nodes and users. They can be set for spots, announces,
+WWV and WCY. They can also be used for hops. An example might look like
+this ...
<P>
<BLOCKQUOTE><CODE>
<PRE>
-##Start DXSpider on bootup and respawn it should it crash
-DX:3:respawn:/bin/su -c "/usr/bin/perl -w /spider/perl/cluster.pl" sysop >/dev/tty7
+accept/spot node_default by_zone 14,15,16,20,33
+set/hops node_default spot 50
</PRE>
</CODE></BLOCKQUOTE>
+<P>This filter is for spots only, you could set others for announce, WWV and WCY.
+This filter would work for ALL nodes unless a specific filter is written to
+override it for a particular node. You can also set a user_default should
+you require. It is important to note that default filters should be
+considered to be "connected". By this I mean that should you override the
+default filter for spots, you need to add a rule for the hops for spots also.
<P>
-<P>This will automatically start DXSpider on tty7 (ALT-F7) on bootup and restart it should it crash for any reason.
-<P>
-<P>As root type the command <EM>telinit q</EM>. DXSpider should start up immediately. You will see the output on tty7
-and if you login as <EM>sysop</EM> you should find everything running nicely.
-<P>
-<P>So far so good, now to automate script connections...
-<P>
-<H3>The crontab file</H3>
+<H2><A NAME="ss3.5">3.5 Advanced filtering</A>
+</H2>
-<P>Login as <EM>sysop</EM> and create a file in /spider/local_cmd called crontab. Edit it with your favourite editor and
-add a line like this (I have included a comment)
+<P>Once you are happy with the results you get, you may like to experiment.
+<P>
+<P>The previous example that filters hf/cw spots and accepts vhf/uhf spots from EU
+can be written with a mixed filter, for example ...
<P>
<BLOCKQUOTE><CODE>
<PRE>
-# check every 10 minutes to see if gb7xxx is connected and if not
-# start a connect job going
-
-0,10,20,30,40,50 * * * * start_connect('gb7xxx') if !connected('gb7xxx')
+rej/spot on hf/cw
+acc/spot on 0/30000
+acc/spot 2 on 50000/1400000 and (by_zone 14,15,16 or call_zone 14,15,16)
</PRE>
</CODE></BLOCKQUOTE>
+<P>Note that the first filter has not been specified with a number. This will
+automatically be assumed to be number 1. In this case, we have said <EM>reject all
+HF spots in the CW section of the bands but accept all others at HF. Also
+accept anything in VHF and above spotted in or by operators in the zones
+14, 15 and 16</EM>. Each filter slot actually has a 'reject' slot and
+an 'accept' slot. The reject slot is executed BEFORE the accept slot.
<P>
-<P>The callsign involved will be the callsign of the cluster node you are going to connect to. This will now check
-every 10 minutes to see if gb7xxx is connected, if it is then nothing will be done. If it is not, then a connect
-attempt will be started.
+<P>It was mentioned earlier that after a reject test that doesn't match, the default
+for following tests is 'accept', the reverse is true for 'accept'. In the example
+what happens is that the reject is executed first, any non hf/cw spot is passed
+to the accept line, which lets through everything else on HF. The next filter line
+lets through just VHF/UHF spots from EU.
<P>
-<P>There are probably lots of other things you could use this crontab file for. If you want to know more about it, look
-at the
-<A HREF="http://www.dxcluster.org/cron.html">DXSpider</A> website at the cron page where it is
-explained more fully.
<P>
<HR>
<A HREF="adminmanual-4.html">Next</A>
projects / spider.git / blobdiff