Problem Cases

Unknown macro: {html}

<H3>Introduction</H3>
This directory contains a set of pages describing various problems, relevant information, how they were tackled and possibly resolved. The date is roughly when the page for this problem was created.

<DL>
<DT>August 17, 2010 LCLS Data Transfers from SLAC to DESY performing poorly</DT>
<DD>Appears it is a host related problem at DESY - UNDERSTOOD
</DD><DT>August 23, 2007 <A href="http://www.slac.stanford.edu/grp/scs/net/case/niit-dns-aug07/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/niit-dns-aug07/">IEPM-BW host at NIIT not reachable</A> Getting two different IP addresses when trying to resolve host name.
</DT><DD>NIIT DNS servers were out of sync and both were down - UNDERSTOOD </DD><DT>July 8, 2007 <A href="http://www.slac.stanford.edu/grp/scs/net/case/comsats-jul07/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/comsats-jul07/">Reporting COMSATS Pakistan not reachable</A>

Routing loop blocked access, unable to report problem.
</DT><DD>Loss of power at COMSats - UNDERSTOOD </DD><DT>July 1, 2007 <A href="http://www.slac.stanford.edu/grp/scs/net/case/skype-jun06/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/skype-jun06/">SLAC Internal Connection</A> Long delays loading SLAC home page on Visitor network using wireless </DT><DD>SKYPE (or other P2P application) was using the host as a supernode - UNDERSTOOD </DD><DT>June 26, 2007:
<A href="http://www.slac.stanford.edu/grp/scs/net/case/slac20070626/slac_external.htm" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/slac20070626/slac_external.htm">SLAC External Connections</A> </DT><DD>Offiste connections to some sites was very poor STILL DIAGNOSING

</DD><DT>December 1, 2006:
<A href="http://www.slac.stanford.edu/grp/scs/net/case/uvic/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/uvic/">
Inadequate throughput between SLAC and U Victoria, Dec 2006</A> </DT><DD>User wants to increase throughput between SLAC and UVic Partially UNDERSTOOD </DD><DT>August 22, 2006:
<A href="http://www.slac.stanford.edu/grp/scs/net/case/bnl-aug06/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/bnl-aug06/">
Problems From BNL August '06'</A>
</DT><DD>BNL unreachable from some places - INPROCESS.
</DD><DT>August 4, 2006:
<A href="http://www.slac.stanford.edu/grp/scs/net/case/tenet-aug06/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/tenet-aug06/">

Problems accessing S. African host 04 August '06'</A> </DT><DD>Denial of Service attack, followed by loss of power - UNDERSTOOD.
</DD><DT>July 25, 2006:
<A href="http://www.slac.stanford.edu/grp/scs/net/case/binp-jul06/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/binp-jul06/">
Problems with BINP-SLAC link July '06</A> </DT><DD>Power problems in Akademogorock region causes many services to be unavailable from SLAC to BINP - UNDERSTOOD.
</DD><DT> May 18, 2006:
<A href="http://www.slac.stanford.edu/grp/scs/net/case/goddard-may06/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/goddard-may06/">Poor throughput for GLAST data transfers between SLAC and NRL Washinton</A>

</DT><DD>Too small TCP windows set at default, firewall limiting throughput, application needs to apply ssh throughput patches, mandated application cannot use multiple streams or big windows, alternate path had port blocking. - UNDERSTOOD </DD><DT>May 14, 2006:
<A href="http://www.slac.stanford.edu/grp/scs/net/case/taiwan-may06/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/taiwan-may06/">Throughput between SLAC and Taiwan</A> </DT><DD>TCP windows and number of parallel styreams needed optimizing. - UNDERSTOOD </DD><DT>May 9, 2006:
<A href="http://www.slac.stanford.edu/grp/scs/net/case/rttanomaly-apr06/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/rttanomaly-apr06/">
Large increase in RTT from South Africa to CERN, April 2006</A>

</DT><DD>Change in routing traffic from S. Africa to CERN from using GEANT to going via Abilene and the U.S. after Abilene started exchanging traffic with S. African TENET - UNDERSTOOD,

</DD><DT>February 1, 2006:
<A href="http://www.slac.stanford.edu/grp/scs/net/case/ornl-jan06/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/ornl-jan06/">
Problems with Oracle between SLAC and ORNL, Jan 2006</A> </DT><DD>Appears to be an application problem - UNDERSTOOD </DD><DT>January 22, 2006:
<A href="http://www.slac.stanford.edu/grp/scs/net/case/sandiego_sunnyvale/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/sandiego_sunnyvale/">
Effect of ESnet routing change between SLAC and SDSC</A> </DT><DD>San Diego Sunnyvale connectivity was interrupted by Level(3) maintenance.
An alternate path was used. This study looks at the impact on the various IEPM metrics of the route change. UNDERSTOOD

</DD><DT>December 28, 2005:
<A href="http://www.slac.stanford.edu/grp/scs/net/case/bnl_dec05/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/bnl_dec05/">

Problems with link to/from BNL</A>
</DT><DD>Multiple paths seen from and to BNL showed step downs in pathchirp performance Believed to be related to the host at BNL - NOT RESOLVED </DD><DT>September 10, 2005: <A href="http://www.slac.stanford.edu/grp/scs/net/case/stanford-sep05/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/stanford-sep05/">
Problems with Stanford's Connectivity</A> </DT><DD>Fan failure in Cisco 15454 causes route to CENIC/Sunnyvale to fail, and switch over to using ESnet.
UNDERSTOOD.

</DD><DT>August 16, 2005: <A href="http://www.slac.stanford.edu/grp/scs/net/case/bnl-aug05/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/bnl-aug05/">
Problems with link to/from BNL</A>
</DT><DD>Fiber cut causes OC48 outage and fail over to a T3 link UNDERSTOOD. </DD><DT>July 1, 2004: <A href="http://www.slac.stanford.edu/grp/scs/net/case/pakjul05/jun-july.htm" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/pakjul05/jun-july.htm">
Fiber outage to Pakistan June 27th to July 8, 2005</A> </DT><DD>Fiber cut off coast of Karachi causes 12 day outage to Pakistan UNDERSTOOD </DD><DT>May 31, 2005: <A href="http://www.slac.stanford.edu/grp/scs/net/case/binp-may05/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/binp-may05/">

New Route from SLAC to BINP Novosibirsk via GLORIAD</A> </DT><DD>See the effect of the new route in improving RTT and throughput UNDERSTOOD. </DD><DT>April 18, 2005: <A href="http://www.slac.stanford.edu/grp/scs/net/case/binp-apr05/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/binp-apr05/">
Loss of Connectivity between SLAC and BINP/Novosibirsk</A> </DT><DD>Some change caused the connectivity between SLAC and BINP to be blocked by firewalls in Novosibirsk.
</DD><DT>September 2, 2004: <A href="http://www.slac.stanford.edu/grp/scs/net/case/i2-sep04/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/i2-sep04/">
Loss of Connectivity from SLAC for about 10 hours</A>

</DT><DD>Due to CENIC maintenance SLAC lost conectivity to many sites from around midnight until 10:30am September 2, 2004.
This is nicely illustrated in traceroutes taken each 10 minute interval to multiple sites from SLAC, and in particular can be nicely visualized by means of a topology map. - UNDERSTOOD. </DD><DT>February 25, 2004: <A href="/grp/scs/net/talk03/niit-jan04.ppt" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/http://www.slac.stanford.edu/grp/scs/net/case/http://www.slac.stanford.edu/grp/scs/net/case/http://www.slac.stanford.edu/grp/scs/net/case/http://www.slac.stanford.edu/grp/scs/net/case/http://www.slac.stanford.edu/grp/scs/net/case/http://www.slac.stanford.edu/grp/scs/net/case/http://www.slac.stanford.edu/grp/scs/net/case/http://www.slac.stanford.edu/grp/scs/net/case/grp/scs/net/talk03/niit-jan04.ppt">
Network performance to Pakistan</A>
</DT><DD>Shows Internet performance from SLAC to a few Pakistani sites,
including: differences in performance depending on providers;
identifying bottlenecks; effects of upgrade for NIIT;
routing between Pakistani sites going via London.

UNDERSTOOD. 
</DD><DT>January 8, 2004: <A href="http://www.slac.stanford.edu/grp/scs/net/case/hangzhou/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/hangzhou/">
Effect of routing on RTTs and losses to 2 sites in Hangzhou, China</A>

</DT><DD>Site with better Internet connectivity had much larger RTTs, but lower losses - UNDERSTOOD. </DD><DT>October 3, 2003: <A href="http://www.slac.stanford.edu/grp/scs/net/case/caltech/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/caltech/">
Throughput from SLAC to Caltech reduced by a factor of 5</A> </DT><DD>CENIC's routing to Caltech was incorrect - RESOLVED. </DD><DT>July 23, 2003: <A href="http://www.slac.stanford.edu/grp/scs/net/case/i2sox/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/i2sox/">
Route to Sox (Southern Crossroads) has been changed, the bandwidth increased in factor 2 </A>

</DT><DD>Abilene routing, traceroute analysis - RESOLVED.

</DD><DT>June 12, 2003: <A href="http://www.slac.stanford.edu/grp/scs/net/case/esnet-route/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/esnet-route/">
Effect of change from Internet2 to
ESnet for the routes from SLAC to Europe</A>
</DT><DD>2 problems, routing for IN2P3, incorrect iperf configuration for CESnet - RESOLVED. </DD><DT>June 10, 2003: <A href="http://www.slac.stanford.edu/grp/scs/net/case/dl-upgrade/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/dl-upgrade/">One of our important partner Daresbury lab (UK) upgraded line to Internet to 1GBit/s </A> </DT><DD>Routing - RESOLVED.

</DD><DT>June 6, 2003: <A href="http://www.slac.stanford.edu/grp/scs/net/case/rtchanges/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/rtchanges/">Routing changes hit many sites in US</A>

</DT><DD>Routing - RESOLVED.
</DD><DT>May 22, 2003: <A href="http://www.slac.stanford.edu/grp/scs/net/case/su-cern/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/su-cern/">SLAC to CERN RTT Anomalies</A> </DT><DD>Routing - RESOLVED. </DD><DT>May 20, 2003: <A href="http://www.slac.stanford.edu/grp/scs/net/case/binp-may03/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/binp-may03/">Loss of connectivity between SLAC and Novosibirsk</A>

</DT><DD>Misconfigured router - RESOLVED. </DD><DT>May 7, 2003: <A href="http://www.slac.stanford.edu/grp/scs/net/case/abilene" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/abilene">

The unusual bandwidth drop to European sites (connected via Abilene) </A> </DT><DT>May 1, 2003: <A href="http://www.slac.stanford.edu/grp/scs/net/case/nersc" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/nersc">
The influence of strong LAN traffic on target hosts in monitoring results (NERSC,ANL)</A> </DT><DD>Effect of local NFS traffic - RESOLVED.
</DD><DT>November 6, 2002: <A href="http://www.slac.stanford.edu/grp/scs/net/case/ncsa" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/ncsa">Change of Routing to NCSA and others</A>.

</DT><DD>Change of routing - RESOLVED </DD><DT>September 20, 2002: <A href="http://www.slac.stanford.edu/grp/scs/net/case/bwe/index.html" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/bwe/index.html">Change of Routing Network connectivity problems - reports from BWE</A>

</DT><DD>SLAC-UK, and SLAC-ANL - RESOLVED </DD><DT>July 26, 2002: <A href="http://www.slac.stanford.edu/grp/scs/net/case/nsk/nsk02.html" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/nsk/nsk02.html">
Network connectivity between SLAC and BINP, Novosibirsk following upgrade</A> </DT><DD>Results of effectiveness of network upgrade - RESOLVED.

</DD><DT>May 30, 2002: <A href="http://www.slac.stanford.edu/grp/scs/net/case/teleglobe/index.html" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/teleglobe/index.html">Change of Routing between UK and US.</A> </DT><DD>Routing Change - RESOLVED

</DD><DT>Mar 21, 2002: <A href="http://www.slac.stanford.edu/grp/scs/net/case/rome-mar02/problem-20020320.html" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/rome-mar02/problem-20020320.html">Poor throughput from SLAC to INFN/Rome</A> </DT><DD>Poor packet loss - RESOLVED </DD><DT>Jan 4, 2002: <A href="http://www.slac.stanford.edu/grp/scs/net/case/bandwidth-tests/kflags-20020105.html" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/bandwidth-tests/kflags-20020105.html">
A sudden change in bandwidth utilization</A>

</DT><DD>Throughput from monitoring host suddenly dropped from several hundreds of Mbits/s to about 100Mbits/s due to turning on debugging flags - RESOLVED </DD><DT>Jan 8, 2002: <A href="http://www.slac.stanford.edu/grp/scs/net/case/in2p3-jan02/problem-20020108.html" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/in2p3-jan02/problem-20020108.html">Problems with performance between SLAC and IN2P3, Jan '02</A> </DT><DD>Only ~ 20Mbits/s throughput achievable on 30Mbps ATM VP bottleneck - RESOLVED

</DD></DL>
<DL>
<DT>Dec 13, 2001: <A href="http://www.slac.stanford.edu/grp/scs/net/case/cern/problem-20011213.html" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/cern/problem-20011213.html">Problems with performance between SLAC and CERN</A> </DT><DD>Large router buffers adversely affect interactive use - RESOLVED

</DD></DL>
<DL><DT>Sep 27, 2001: <A href="http://www.slac.stanford.edu/grp/scs/net/case/rl/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/rl/">Nework connectivity between SLAC and RL, UK</A> </DT><DD>Poor performance from SLAC to Rutherford Lab, UK caused by firewall and small windows - RESOLVED </DD><DT>Sep 19, 2001: <A href="http://www.slac.stanford.edu/grp/scs/net/case/ihepsu/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/ihepsu/">Problems with links from SLAC to IHEP.SU</A>

</DT><DD>High packet loss prevents ssh access, due to host problems - RESOLVED </DD><DT>Sep 11, 2001: <A href="http://www.slac.stanford.edu/grp/scs/net/case/wtc/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/wtc/">Impact of World Trade Center & Pentagon attacks on Internet performance</A>

</DT><DD>How did the WTC collapse affect Internet connectivity for SLAC - RESOLVED </DD><DT>May 3, 2001: <A href="http://www.slac.stanford.edu/grp/scs/net/case/qbss/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/qbss/">Qbone Scavenger Service at SLAC</A> </DT><DD>Need to find hosts/sites to try with QBone Scavenger services - RESOLVED </DD><DT>Mar 28, 2001: <A href="http://www.slac.stanford.edu/grp/scs/net/case/international/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/international/">International Internet connectivity from SLAC</A>

</DT><DD>Request for information on quality of International links - RESOLVED </DD><DT>Mar 19, 2001: <A href="http://www.slac.stanford.edu/grp/scs/net/case/ihep-mar01/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/ihep-mar01/">Comparing ihep.ac.cn with cas.ac.cn - March 2001</A>

</DT><DD>Request to understand the proposed effect of changing an ISP - RESOLVED </DD><DT>Mar 1, 2001: <A href="http://www.slac.stanford.edu/grp/scs/net/case/su-dsl-mar01/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/su-dsl-mar01/">Packet loss on Stanford-Pac Bell DSL links - Mar 01</A> </DT><DT>Feb 25, 2001: <A href="http://www.slac.stanford.edu/grp/scs/net/case/su-dsl-feb01/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/su-dsl-feb01/">Packet loss on Stanford-Pac Bell DSL links - Feb 01</A> </DT><DD>Problems with users DSL service due to bad pairs to home - RESOLVED

</DD><DT>Jan 27, 2001: <A href="http://www.slac.stanford.edu/grp/scs/net/case/kangadata02/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/kangadata02/">Throughput between Tersk03 and Kangadata</A> </DT><DD>Problems in host to host throughput on LAN due to windows being to small - RESOLVED

</DD><DT>Jan 22, 2001: <A href="http://www.slac.stanford.edu/grp/scs/net/case/in2p3-jan01/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/in2p3-jan01/">Network connectivity between SLAC and IN2P3, Jan 2001</A> </DT><DD>Problems with throughput between SLAc and IN2P3 due to misconfigured ATM/PVC - RESOLVED </DD><DT>Jan 12, 2001: <A href="http://www.slac.stanford.edu/grp/scs/net/case/platform/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/platform/">Network connectivity between SLAC and Platform Computing</A> </DT><DD>Problems between SLAC and a commercial site due to poor routing - RESOLVED

</DD><DT>Dec 18, 2000: <A href="http://www.slac.stanford.edu/grp/scs/net/case/utdallas/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/utdallas/">Network connectivity between SLAC and UT Dallas</A> </DT><DD>Multiple probelms with throughput between SLAC and UTDallas due to small windows and packet loss - RESOLVED WINDOW PROBLEM

</DD><DT>Dec 07, 2000: <A href="http://www.slac.stanford.edu/grp/scs/net/case/in2p3" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/in2p3">Network connectivity between SLAC and IN2P3, Lyon, France</A> </DT><DD>User question on suitability of network connections to BaBar tier A regional computing site - RESOLVED </DD><DT>Dec 06, 2000: <A href="http://www.slac.stanford.edu/grp/scs/net/case/nsk/" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/nsk/">Network connectivity between SLAC and BINP, Novosibirsk</A> </DT><DD>Investigation of poor throughput from SLAC to Novosirbirsk, resulting in successful recommendation for link upgrade - RESOLVED

</DD><DT>Sep 11, 2000: <A href="/comp/net/wan-mon/slac-caltech.html" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/http://www.slac.stanford.edu/grp/scs/net/case/http://www.slac.stanford.edu/grp/scs/net/case/http://www.slac.stanford.edu/grp/scs/net/case/http://www.slac.stanford.edu/grp/scs/net/case/http://www.slac.stanford.edu/grp/scs/net/case/http://www.slac.stanford.edu/grp/scs/net/case/http://www.slac.stanford.edu/grp/scs/net/case/http://www.slac.stanford.edu/grp/scs/net/case/comp/net/wan-mon/slac-caltech.html">Caltech to SLAC thruput</A> </DT><DD>Poor throughput from Caltech to SLAC while SLAC to Caltech was OK, partially resolved by moving from ESnet to Internet 2 - PARTIALLY RESOLVED

</DD><DT>Sep 03, 2000: <A href="/comp/net/wan-mon/slac-ilan.html" mce_href="http://www.slac.stanford.edu/grp/scs/net/case/http://www.slac.stanford.edu/grp/scs/net/case/http://www.slac.stanford.edu/grp/scs/net/case/http://www.slac.stanford.edu/grp/scs/net/case/http://www.slac.stanford.edu/grp/scs/net/case/http://www.slac.stanford.edu/grp/scs/net/case/http://www.slac.stanford.edu/grp/scs/net/case/http://www.slac.stanford.edu/grp/scs/net/case/http://www.slac.stanford.edu/grp/scs/net/case/comp/net/wan-mon/slac-ilan.html">SLAC to ILAN (Israel) thruput</A> </DT><DD>Poor performance between SLAC and Israel caused by ESnet upgrade to router using incorrect BGP configuration - RESOLVED <#comment><#comment></#comment> </DD><DT>May 16, 2000: <A href="stanford.html" mce_href="stanford.html">Stanford to SLAC file transfer problems</A> </DT><DD>Poor throughput between SLAc and Stanford campus, required resetting an ATM interface - RESOLVED

</DD><DT>May 09, 2000: <A href="http://www-iepm.slac.stanford.edu/monitoring/qos/uk.html" mce_href="http://www-iepm.slac.stanford.edu/monitoring/qos/uk.html">
UK to US QoS Testing</A>

</DD><DT>May 23, 1999: <A href="http://www.slac.stanford.edu/comp/net/wan-mon/pathology-eg.html" mce_href="http://www.slac.stanford.edu/comp/net/wan-mon/pathology-eg.html">Ping Measurement Pathology Examples</A> </DT><DD>Pings from Linux to WNT on a LAN with normal RTT <

10 msec, manifest an unusual regularity in the sequence number separation of the pings with > 10msec. Probably an OS effect.

RESOLVED.
</DD></DL>

Child pages

Problem Cases