Thursday, October 11, 2007

rPath down again for several hours

About a month ago rBuilder suffered from a rolling repository outage. The infrastructure problems continued this morning when large parts of rPath's infrastructure went down. This may or may not be related to the hour of maintenance conducted between 9pm and 10pm EST on rBuilder Online yesterday. But before 4.30am this morning (EST) large parts of rPath's web services were down according to Antonio Meireles, due to a bad proxy. Services were restored around 7.21am.

A quick look at rPath's infrastructure looks like it is just some colo sitting on some Cogent bandwidth. Nothing wrong with Cogent, we use Cogent in some places, but we're multi-homed and multi-site. A quick DNS lookup on www.rpath.com shows its on 38.100.0.24, the first IP in this block used for rPath appears to be 38.100.0.19 (colo-admin.rpath.com), with 38.100.0.28 appearing to be the last used (at least with configured reverse dns). Hardly enterprise grade, no redundancy, perhaps they should read o3 magazine! :)

Live logs from #conary:

[05:56:22] doniphon > large parts of rPath web down. bad proxy, etc.
[05:59:00] iwilson_ > I saw that
[05:59:11] iwilson_ > ironically enough, they die when I'm searching for something.
[05:59:16] tpfennig > oh downloads are slow today...
...
[06:06:14] doniphon > tpfennig: rBO is flacky atm. anyway fill a bug. *that* should not happen atm.
...
[07:09:35] doniphon > msw Up2 mkj jtate SM2k gxti *@rPath. wiki is down
[07:16:56] SM2k > doniphon: we're working on it
...
[07:21:19] msw > back///

No comments: