Notices
"The 33"-Road Bike Racing We set this forum up for our members to discuss their experiences in either pro or amateur racing, whether they are the big races, or even the small backyard races. Don't forget to update all the members with your own race results.

My geek thread

Old 03-08-11, 03:54 PM
  #576  
My idea of fun
 
kensuf's Avatar
 
Join Date: Jun 2006
Location: Gainesville, FL
Posts: 9,920

Bikes: '06 Litespeed Tuscany, '02 Kona Lavadome, '07 Giant TCR Advanced, '07 Karate Monkey

Liked 59 Times in 36 Posts
HVAC update...

A month ago, I worked with the campus PPD A/C super to schedule a day when we could shut everything in the machine room down so that one of his crews could replaced the chilled water pump. That date was set for today, Tuesday, March 8th, 2011, beginning at 8AM.

Last week I got an email confirmation on the date / time for the schedule from PPD. There was a slight error on the start time (they had 7), so I emailed back with a "are you sure?". Their person wrote back saying that indeed the work would begin at 8, but they were emailing us announcing the window beginning at 7.

No problem. I wake up this morning at 5:30, come to work and begin shutting down 6 racks worth of server equipment. That includes 10 external disk arrays (total formatted usable capacity of around 80tb, raw in the 160tb range).

8:00 rolls around, no sign of PPD. I figure these guys are basically state workers, so I'm not overly surprised..

8:45, still no sign of PPD. I go grab a coffee.

9:15, still no sign of PPD. At that point I call over there and ask where the work crew is that's supposed to be changing the chilled water pump.

After a few minutes, I get a call back, "the work crew came over there and did the work yesterday."

"WHAT!?!?! You mean you guys shut down my chilled water and A/C on the 7th without notifying us? We have close to $500k worth of temperature sensitive equipment in there, and millions of dollars worth of research data! We spent weeks working out the outage and you guys did it a day in advance?"

"Can you hold on? Let me double check on that. Can we call you back?"

Five minutes later, some bubba calls me back. "Well sir, it was such a nice day yesterday, we figured we could go ahead and knock that project out and it wouldn't have been too much trouble."

HOLY CRAP.

We have thermal alarms on the equipment, and nothing tripped, but what if it had? Would these guys have even begun to know what to do if the audible alarms started going off? Would they have known who to contact? Can we say no?

Needless to say, we're scheduling a meeting with the A/C super, a director of PPD, myself, the chair of my department, and possibly the dean of my college that oversees infrastructure..

(so to tie this into the failure / training threads, I've been basically a mental zombie since around 10:30 this morning)
kensuf is offline  
Old 03-08-11, 04:05 PM
  #577  
Making a kilometer blurry
 
waterrockets's Avatar
 
Join Date: May 2006
Location: Austin (near TX)
Posts: 26,170

Bikes: rkwaki's porn collection

Liked 92 Times in 38 Posts
Originally Posted by mattm
iPhone seems to handle the whole versioning thing much better.

Developing for Android, if you only build for the latest bits you're cutting out like 80% of possible customers.. and to build for those other 80% you have to use different version of the API, different feature sets. It's a nightmare, or at least it seemed like too much of a nightmare for this hobby-type dev who wanted to write some droid apps.

Fwiw I don't own or use a cell phone - just some observations.
Yep. That's the way it felt when I was developing PC games. You had to have a "minimum system" spec, but still push all the latest hardware if a customer had it. Scalable games blow.

When I switched to console (XBox, PS2, GameCube, etc) life got a lot easier.

FWIW: most Android app ideas don't require anything beyond Android 1.5 or so. Some of the higher-tech stuff does, but those apps don't make any money anyway (too long to develop stuff like AR when you look at how few people give a crap). That said, I've written AR that runs on 1.5...

Originally Posted by kensuf
"WHAT!?!?! You mean you guys shut down my chilled water and A/C on the 7th without notifying us?
Holy crap.
waterrockets is offline  
Old 03-08-11, 04:07 PM
  #578  
**** that
 
mattm's Avatar
 
Join Date: Dec 2006
Location: CALI
Posts: 15,402
Liked 104 Times in 30 Posts
Originally Posted by waterrockets
Holy crap.
+0x10000000!

Major ouch.
__________________
cat 1.

my race videos
mattm is offline  
Old 03-08-11, 04:09 PM
  #579  
BatŁwŁ Creakcreak
 
Join Date: Jan 2007
Location: The illadelph
Posts: 20,798
Liked 289 Times in 161 Posts
Damn dude!

Your worries are much greater than mine, but I had a headache today as well.

The program for the online conference that I run had a hiccup today and it left 10+ sites from all over the world staring at a blank screen for half an hour while I struggled to figure out what was going on. Unbeknown to me, the server has a mirror which is what I accessed that day and it created two mirror conferences with the same ID's which were impossible to tell apart.

I think they're making an excuse on their end and my boss is super frustrated with the system (as am I because I get blamed for it, even though this was totally out of my control). The words "1/3 of your salary comes out of a grant that funds these conferences, and if we give these responsibilities to xyz, then you're not going to work for 2/3 the pay" left his mouth.

Needless to say, I'm looking around at other options. Just.In.Case.
ridethecliche is offline  
Old 03-08-11, 04:14 PM
  #580  
Draught
 
jwible's Avatar
 
Join Date: Jul 2005
Location: Georgia
Posts: 4,051

Bikes: N-1 where N = number needed

Likes: 0
Liked 0 Times in 0 Posts
I heard a thing on NPR, I think it was, the other day about airlines getting rid of paper approach maps and having pilots use iPads instead. This frightens me if the pilots are also not required to keep paper charts on board. In private pilot check rides one of the first things the inspector does is remove the batteries from any flight calculators you may have.
jwible is offline  
Old 03-08-11, 04:19 PM
  #581  
BatŁwŁ Creakcreak
 
Join Date: Jan 2007
Location: The illadelph
Posts: 20,798
Liked 289 Times in 161 Posts
Originally Posted by waterrockets
Yep. That's the way it felt when I was developing PC games. You had to have a "minimum system" spec, but still push all the latest hardware if a customer had it. Scalable games blow.

When I switched to console (XBox, PS2, GameCube, etc) life got a lot easier.

FWIW: most Android app ideas don't require anything beyond Android 1.5 or so. Some of the higher-tech stuff does, but those apps don't make any money anyway (too long to develop stuff like AR when you look at how few people give a crap). That said, I've written AR that runs on 1.5...
The computer that I got is a little excessive for what I need. But the price was right and the specs are grand.

Think, Upper mid tier gaming computer with a 23 inch HD monitor. My parents gave me a budget, so I used it to the max and am paying for part of the stuff myself. Think hexa core amd with a 512 mb or 1gb graphics card that's about mid range in terms of specs. I can't wait to get a good FPS for the computer. Though the first thing I'm going to do is install HL-2 and get CS-Source. It'll be excessively wonderful

Then my mom tried to give me money when I left home. I love her dearly but I refused. I told her that they were buying me a computer so I had more than my share of what was needed. I think the drugs got to her.

Wr, I understand what you mean for consoles. I bet it's MUCH easier there which is why many are claiming the 'death' of the PC as a game system. Sure the high end is higher, but it's annoying for developers and game consoles are awesome now-a-days. That's why I bought one, so I didn't have to worry about having a constantly up to date PC. The opportunity presented itself though, and so now I have a computer that should be game worthy for 2ish years and be fine for 'regular computer' use for atleast 4-5. I'm content.

Now if only the PS3 played PS2 games
I might have to steal my PS2 back from the sister once I'm done studying. There are too many good RPG's on that console (and the PS1) that I would love to play.
ridethecliche is offline  
Old 03-08-11, 04:20 PM
  #582  
BatŁwŁ Creakcreak
 
Join Date: Jan 2007
Location: The illadelph
Posts: 20,798
Liked 289 Times in 161 Posts
Originally Posted by jwible
I heard a thing on NPR, I think it was, the other day about airlines getting rid of paper approach maps and having pilots use iPads instead. This frightens me if the pilots are also not required to keep paper charts on board. In private pilot check rides one of the first things the inspector does is remove the batteries from any flight calculators you may have.
They justified this by saying that the iPad can be reset in a matter of a minute max. If they have another one on board, then that gets rid of the problem all together. If you're worried about water, etc. Maps would be about as useful as an IPAD.

I think it's efficient, not to mention neat. They'll probably keep the maps onboard planes as a backup anyway. The planes are run by computers anyway, so if you're worried about that then...
ridethecliche is offline  
Old 03-08-11, 04:42 PM
  #583  
Making a kilometer blurry
 
waterrockets's Avatar
 
Join Date: May 2006
Location: Austin (near TX)
Posts: 26,170

Bikes: rkwaki's porn collection

Liked 92 Times in 38 Posts
Originally Posted by ridethecliche
They justified this by saying that the iPad can be reset in a matter of a minute max. If they have another one on board, then that gets rid of the problem all together. If you're worried about water, etc. Maps would be about as useful as an IPAD.

I think it's efficient, not to mention neat. They'll probably keep the maps onboard planes as a backup anyway. The planes are run by computers anyway, so if you're worried about that then...
I still don't buy it as the only source. There are too many opportunities for bugs in such an application. They'd be fine most of the time, but it's the corner cases that concern me.

Regarding maps and water, there are fiberglass maps that won't tear can be read under water. Not sure what aeronautical charts are made from though.
waterrockets is offline  
Old 03-08-11, 05:06 PM
  #584  
**** that
 
mattm's Avatar
 
Join Date: Dec 2006
Location: CALI
Posts: 15,402
Liked 104 Times in 30 Posts
Originally Posted by waterrockets
I still don't buy it as the only source. There are too many opportunities for bugs in such an application. They'd be fine most of the time, but it's the corner cases that concern me.

Regarding maps and water, there are fiberglass maps that won't tear can be read under water. Not sure what aeronautical charts are made from though.
Corner cases can definitely be interesting.

There was one fine example of a new fighter jet, that when it crossed the equator the plane would flip upside down every time.... now that's a bad corner case! Though the equivalent here would be the map flipping upside down, but still, the idea scares me.
__________________
cat 1.

my race videos
mattm is offline  
Old 03-08-11, 05:09 PM
  #585  
Making a kilometer blurry
 
waterrockets's Avatar
 
Join Date: May 2006
Location: Austin (near TX)
Posts: 26,170

Bikes: rkwaki's porn collection

Liked 92 Times in 38 Posts
Originally Posted by mattm
when it crossed the equator the plane would flip upside down every time


Exactly.
waterrockets is offline  
Old 03-08-11, 05:20 PM
  #586  
Killing Rabbits
 
Join Date: Apr 2005
Posts: 5,697
Liked 217 Times in 102 Posts
Oh no a computer almost got too warm! In my experience complaints about potential damage to scientific equipment are going to fall on deaf ears. If you want something to be done it has to involve public relations or safety issues. “Holy crap” events involve HazMat teams and building evacuations….

In this old building I’m happy so long as water isn’t leaking directly onto our equipment and even that is often too much to ask.

Lastly could all that data really have been lost? If so, you guys have bigger problems.
Enthalpic is offline  
Old 03-08-11, 05:46 PM
  #587  
Senior Member
 
shovelhd's Avatar
 
Join Date: Jun 2010
Location: Western MA
Posts: 15,669

Bikes: Yes

Likes: 0
Liked 0 Times in 0 Posts
kensuf, I'd say unbelieveable, but I've been in this business too long. At least the reserve capacity handled the temp rise. If we lose a chiller it goes to 95 degrees within 20 minutes and an alarm calls my cell.
shovelhd is offline  
Old 03-08-11, 06:05 PM
  #588  
My idea of fun
 
kensuf's Avatar
 
Join Date: Jun 2006
Location: Gainesville, FL
Posts: 9,920

Bikes: '06 Litespeed Tuscany, '02 Kona Lavadome, '07 Giant TCR Advanced, '07 Karate Monkey

Liked 59 Times in 36 Posts
Originally Posted by Enthalpic
Oh no a computer almost got too warm! In my experience complaints about potential damage to scientific equipment are going to fall on deaf ears. If you want something to be done it has to involve public relations or safety issues. “Holy crap” events involve HazMat teams and building evacuations….

In this old building I’m happy so long as water isn’t leaking directly onto our equipment and even that is often too much to ask.

Lastly could all that data really have been lost? If so, you guys have bigger problems.
I guess you have no idea the amount of time and manpower that could be lost while we recovered everything.
kensuf is offline  
Old 03-08-11, 07:18 PM
  #589  
Killing Rabbits
 
Join Date: Apr 2005
Posts: 5,697
Liked 217 Times in 102 Posts
I'm not saying your problem wasn't real or serious, just that it won't be a big deal to others.

We had a HVAC issue (fume hoods) that disabled ~30 employees for 7 months straight. I thought heads would roll over that, they didn't.
Enthalpic is offline  
Old 03-08-11, 07:19 PM
  #590  
Senior Member
 
shovelhd's Avatar
 
Join Date: Jun 2010
Location: Western MA
Posts: 15,669

Bikes: Yes

Likes: 0
Liked 0 Times in 0 Posts
Originally Posted by kensuf
I guess you have no idea the amount of time and manpower that could be lost while we recovered everything.
I don't know the details of your operation, but I have been there during the aftermath of 9/11. It sucked on so many levels.
shovelhd is offline  
Old 03-08-11, 10:53 PM
  #591  
Making a kilometer blurry
 
waterrockets's Avatar
 
Join Date: May 2006
Location: Austin (near TX)
Posts: 26,170

Bikes: rkwaki's porn collection

Liked 92 Times in 38 Posts
Agreed that most people don't understand the costs involved in recovery. If it happens once or twice, it's quite an education.
waterrockets is offline  
Old 03-08-11, 11:02 PM
  #592  
Lula Mae = 15 lbs.
 
bike_boy's Avatar
 
Join Date: Dec 2007
Location: I keep hitting <Escape>, but I'm still here!
Posts: 766

Bikes: 2011 Trek Madone 6.5 w/SRAM Red 22

Likes: 0
Liked 0 Times in 0 Posts
Originally Posted by waterrockets
Agreed that most people don't understand the costs involved in recovery. If it happens once or twice, it's quite an education.
Ah, RTO & RPO...
bike_boy is offline  
Old 03-08-11, 11:47 PM
  #593  
Making a kilometer blurry
 
waterrockets's Avatar
 
Join Date: May 2006
Location: Austin (near TX)
Posts: 26,170

Bikes: rkwaki's porn collection

Liked 92 Times in 38 Posts
Today I think I nailed my Android offline maps solution to scale imagery for missing zoom levels. Just need to test it tomorrow. The Mercator projection quad square tile system is really sweet. Math is pretty straightforward, and so is tile storage.

Just like the augmented reality stuff, management is going to need to pull me off these tasks, because I'm about to go all crazy with math-happy feature creep.

Last edited by waterrockets; 03-08-11 at 11:56 PM.
waterrockets is offline  
Old 03-09-11, 06:30 AM
  #594  
My idea of fun
 
kensuf's Avatar
 
Join Date: Jun 2006
Location: Gainesville, FL
Posts: 9,920

Bikes: '06 Litespeed Tuscany, '02 Kona Lavadome, '07 Giant TCR Advanced, '07 Karate Monkey

Liked 59 Times in 36 Posts
Originally Posted by shovelhd
I don't know the details of your operation, but I have been there during the aftermath of 9/11. It sucked on so many levels.
Oof. Yeah I can't imagine: as a bystander more than 1000 miles away, that sucked on just an emotional level. Rolling up shirt sleeves and getting to work on bringing things back must have been tough on you guys.
kensuf is offline  
Old 03-09-11, 07:07 AM
  #595  
Senior Member
 
shovelhd's Avatar
 
Join Date: Jun 2010
Location: Western MA
Posts: 15,669

Bikes: Yes

Likes: 0
Liked 0 Times in 0 Posts
Originally Posted by kensuf
Oof. Yeah I can't imagine: as a bystander more than 1000 miles away, that sucked on just an emotional level. Rolling up shirt sleeves and getting to work on bringing things back must have been tough on you guys.
Fortunately, we had 321 people in the North Tower, and 321 people got out. By 11:30am we were going 24x7 setting up temporary work spaces, networks, and rolling DR recovery for their locally stored data that was obviously destroyed. Unfortunately, it was my 2nd day on the job. I'm not afraid to say that I was scared sh*tless. We got it done, though, and they were up and running in 48 hours in an abandoned shopping mall. This is why you spend millions on backup, recovery, replication, and DR plans and drills, and why that effort can never really be appreciated until you need it.
shovelhd is offline  
Old 03-31-11, 10:12 AM
  #596  
Making a kilometer blurry
 
waterrockets's Avatar
 
Join Date: May 2006
Location: Austin (near TX)
Posts: 26,170

Bikes: rkwaki's porn collection

Liked 92 Times in 38 Posts
♪ ♫ I found the ♪ ♫ memory leak ♪ ♫
♪ ♫ I found the ♪ ♫ memory leak ♪ ♫

waterrockets is offline  
Old 03-31-11, 06:22 PM
  #597  
**** that
 
mattm's Avatar
 
Join Date: Dec 2006
Location: CALI
Posts: 15,402
Liked 104 Times in 30 Posts
Originally Posted by waterrockets
♪ ♫ I found the ♪ ♫ memory leak ♪ ♫
♪ ♫ I found the ♪ ♫ memory leak ♪ ♫

[thecat]
Hopefully you didn't have to find it via code review.. they make apps for that!

So, how did you track it down?
__________________
cat 1.

my race videos
mattm is offline  
Old 03-31-11, 09:35 PM
  #598  
Making a kilometer blurry
 
waterrockets's Avatar
 
Join Date: May 2006
Location: Austin (near TX)
Posts: 26,170

Bikes: rkwaki's porn collection

Liked 92 Times in 38 Posts
Eclipse MAT --> List GC Roots.

Turns out Google Maps Android API has several fields that hang onto the parent Activty. Some other forums had the common bits listed and pulled out with reflection (no public access). Also realized that I wasn't unsubscribing from a listener that kept a reference to the callback method. I thought it was going to be a lot worse, but it only took a day to get it all cleaned up.

I hadn't used MAT before, so that was a learning effort that went well. At the game studio, we had our own badass logger that maintained the same interface for PC, XBox, PS, and Nintendo versions of the game. You could even rewind the stack time. So if a grenade exploded in a room, and the game crashed, you could rewind to when the grenade was created and examine the heap at that time. Really cool.

MAT is a fine tool though. I like it.

I need to start taking a look at IntelliJ though...

Last edited by waterrockets; 03-31-11 at 09:39 PM.
waterrockets is offline  
Old 03-31-11, 09:43 PM
  #599  
Making a kilometer blurry
 
waterrockets's Avatar
 
Join Date: May 2006
Location: Austin (near TX)
Posts: 26,170

Bikes: rkwaki's porn collection

Liked 92 Times in 38 Posts
So, I also found a bug today with the Spidey Sense and no source code. Let's see who can feel it too (kind of an easy one, but somehow I was the only engineer out of 10 or so to pick it out in the three minutes after I heard about it).

Phone app hits a server app. We have a built pair running on the Internet, and a different built pair running on a private network. Slightly different builds on both sides, for both apps.

For some reason, today, right before an important demo, both the Internet and private network systems failed to take new reports made from the phone. They both worked flawlessly yesterday, and continuously for the past several weeks. These two systems don't touch each other in any way at all, and they have completely different usage history and data, but they failed at the same time.

So, what's changed that sprung the bug? (e.g. what's it related to?)
waterrockets is offline  
Old 04-01-11, 09:25 AM
  #600  
Draught
 
jwible's Avatar
 
Join Date: Jul 2005
Location: Georgia
Posts: 4,051

Bikes: N-1 where N = number needed

Likes: 0
Liked 0 Times in 0 Posts
Phone radio turned off. Server IP changed. Server build reverted.
jwible is offline  

Thread Tools
Search this Thread

Contact Us - Archive - Advertising - Cookie Policy - Privacy Statement - Terms of Service - Your Privacy Choices -

Copyright © 2024 MH Sub I, LLC dba Internet Brands. All rights reserved. Use of this site indicates your consent to the Terms of Use.