Many of Geoff's RSS sources frequently change servers, sites, domains, etc, resulting in changed DNS entries, but the DNS entries were being cached indefinitely in CF, so the CFHTTP calls would timeout or fail when the DNS entries were updated (cached didn’t match new real entries). Those RSS sources were therefore lost from his aggregator until Geoff bounced his ColdFusion server, and the caused the DNS cache to be repopulated.
I asked Tom Jordahl take a look, and he discovered there's actually some Sun VM behavior at play here. Tom blogged this earlier today here, but to recap, Java has a default "caching forever" behavior for DNS lookups. See the Sun docs on this subject here for more details: http://java.sun.com/j2se/1.4.2/docs/api/java/net/InetAddress.html
ColdFusion HTTP uses the java.net.InetAddress class to resolve hostnames. While this is the only place in our ColdFusion code to directly use this direct lookup technique, be aware that DB drivers and other OEM technologies may use this, however, and you should therefore keep in mind that this is a server-wide setting, so be sure to test thoroughly before making any changes to a production server.
As it turns out, there is a JVM property to override the default caching behavior, to help thwart DNS spoofing. We're investigating whether to change this default setting in the "Server" configuration to something more reasonable for Blackstone, with some documentation so J2EE setup folks know about this as well and can make the change if they desire this behavior.
Two Java security properties control the TTL values used for positive and negative host name resolution caching:
1) networkaddress.cache.ttl (default: -1). This indicates the caching policy for successful name lookups from the name service. The value is specified as an integer to indicate the number of seconds to cache the successful lookup. A value of -1 indicates "cache forever". A more reasonable default of, say, 14400 or 4 hours (4*60*60) or even a smaller value (30 minutes?) might make more sense in some cases, like Geoff’s, especially where a large number of small public Internet sites are constantly processed via CFHTTP.
2) networkaddress.cache.negative.ttl (default: 10 seconds) This indicates the caching policy for un-successful name lookups from the name service. The value is specified as an integer to indicate the number of seconds to cache the failure for un-successful lookups. A value of 0 indicates "never cache". A value of -1 indicates "cache forever".
To change this behavior on a ColdFusion MX-based server on a Sun 1.4.x or later VM, you’d change these two values in the java.security file in the cfusionmx\runtime\jre\lib\security directory.