Large ISPs with multiple IP blocks full of servers can be a problem. Most of them spew as much spam as legitimate email becuse their accounts and end users are either compromised or are configured to forward messages to other accounts (perhaps accounts on your system?). The result of such forwarding, for example, is that any spam that arrives at the large ISP mailbox is retransmitted by that ISP's servers to the new mailbox.

If a reputation is generated for the IPs of the large ISP's servers then that reputation will represent the averaged behavior of all of the message sources that have to be handled by that ISP. These days such an averaged behavior metric is only slightly better than that of a spam bot!

It would be nice to skip past all of the ISP's servers in the received headers, but those ISPs seldom (more like never) publish the IPs of their servers and even if they did it would be a big management task keeping up with the constant changes.

Drilldown provides a mechanism for clearing up this mess by allowing SNF to learn to see past the large ISP's servers to the original message source. Most of the time, the large ISP's PTR records will contain their domain or something very predictable that will appear in the Received headers of the messages they send. Using drilldown header directives you can automatically train SNF to see past these servers and drill down to the original source for the message. This gives your system more accurate statistics, so that more messages can be blocked based on these IPs without penalizing the ISP's servers.

Note: Some ISPs don't provide received headers for the original sender's IP. Thankfully some of those ISPs do provide the IP in another header. We have a <source/> feature for those cases.

Suppose some large ISP uses the domain mixed-source.net, then you might see received headers similar to:

Received: from out56.mixed-source.net [12.34.56.78] by my0wn1.bestfilterever.net
    (1.2.3.4 / 5.6.7.8) so forth and so on;
Received: from inside34.mixed-source.net [210.1.2.34] by outside56.mixed-source.net
    (and so forth) and so on;
Received: from border124.mixed-source.net [210.1.2.124] by inside34.mixed-source.net
    (and so forth) and so on;
Received: from ugly-spambot-customer.dyn-dsl123.eviltown.cpe9.example.com [99.88.77.66] by
    border124.mixed-source.net (and so forth) and so on;

Then your <drilldown> section might look like this:

<drilldown>
    <received ordinal='0' find='.mixed-source.net [' />
    <received ordinal='1' find='.mixed-source.net [210.' />
    <received ordinal='2' find='.mixed-source.net [210.' />
</drilldown>

The top received header (ordinal 0) created by your system would match the first drilldown header directive and so 12.34.57.78 would be added to GBUdb with the Ignore flag set. SNF would keep looking.

The next received header (ordinal 1) created by mixed-source would match the second drilldown header directive and so 210.1.2.34 would be added to GBUdb with the Ignore flag set. SNF would keep looking.

The next received header (ordinal 2) created by mixed-source would match the third drilldown header directive and so 210.1.2.124 would be added to GBUdb with the Ignore flag set. SNF would keep looking.

Finally, the next received header would not match any header directives. The previous received headers have all been made ineligible as message sources. As a result the IP 99.88.77.66 will be treated as the source IP for this message.

The end result is that messages flowing through the mixed-source ISP will be evaluated based on their content and the original source (not the ISP's servers).

Please email support@armresearch.com with any questions.