Saturday, February 25, 2006
SAN woes
Normally I try not to write about the things I am actually doing at work for paying clients. I tend to not want to give away my own or institutional knowledge for which my firm should charge clients. That's why I stick to discussions about collaboration technologies (OneNote, SharePoint, document management) and personal technology (Treo, TabletPC, Firefox, Microsoft Media Center).
But I've been getting frustrated designing for Network Applications storage as part of my current project. And my prior engagement involved in part carving up storage for Windows servers on an EMC Symmetrix. And I've talked with clients about offerings from Compellent, Falconstor, Hitachi, and HP over the years. I guess I would call my market the middle tier of storage; mainframes are not used and only the occasional Unix system is ever connected to these boxes. Mainly they are for Exchange or Notes clusters, Microsoft SQL server clusters, file clusters and/or NAS, VMware ESX servers, and Windows server boot drives. Ideally the storage controller is used as a mechanism to replicate these systems to another site for DR purposes.
Having worked with some of the major vendors in this middle tier I need to speak my frustration. I do not profess to be a storage engineer so I would not mind any corrections, differences of opinion, or even personal attacks by commenters for this post. I am merely expressing my frustration that nothing is perfect is the small slice of the storage world that I play in. Here are my frustrations:
Network Appliance
This is theoretically the new darling of mid-tier storage, but I grow frustrated with their architecture. Their strength is NAS and in many ways NAS is quite appealing. Consolidating some servers? Want to do it without reconfiguring all your apps? Just have the NAS pretend to be FS1, FS2, NYFS18, whatever. Also, you need far fewer NAS heads than file servers to serve the same amount of data. And, in the NetApp world, snapshots of file systems on NAS are far smaller than snapshots of block storage. And you just have the sense that a custom Linux kernel designed for serving files will do so at a faster rate and far more reliably for far longer than a Windows server.
But, you still need some features that require separate servers when you choose NAS. File share virus protection requires a separate server which must hook into reads and/or writes of every file. Yep, the limiting factor on file serving speeds just became Windows! You could argue that if you protect against viruses via desktop scanning, email gateways, and network edge scanning, you're safe enough, but you still have foreign devices that come onto your network. You could also argue that permissions auditing on a NAS system is not nearly as complete as what you would get in Windows (assuming you set that stuff up). And, in the case of my current client, no NAS can run the docsntss.exe service that DOCS Open requires to properly secure documents. It's a law firm, old document management issue, but if you have that issue, you can't use NAS for your documents. If you were also thinking of using something more sophisticated than RoboCopy or XCOPY /D to replicate data out to branch offices, and you didn't want to put a NetApp NAS in every branch office because you were, I don't know, fiscally responsible or something, then you might want to use FRSv2 in the new R2 release of Windows Server 2003 to replicate file data. Again, that ain't happening with NAS. It's not that I don't still believe in NAS (though you'd have to show me more sophisticated things coming from other vendors, such as running real-time scanning on the NAS head itself) but in my current assignment we've decided against it and in favor of clustered file servers.
However, it is the provision of block storage via iSCSI where I find NetApp falling short. I know that NetApp's strength is that you can make a big-ass RAID group and share all the spindles with a variety of applications. So they're ahead of EMC (Clariion) in my opinion in that they actually can do this. But having RAID4 or RAID-DP be your only options here are wreaking havoc in assigning spindles to write-intensive applications like Exchange and SQL. Maybe I'm gullible or an idiot, but I was given, and am sticking with, a write penalty of 6 for RAID-DP. That means that a group of servers that as a whole are performing 3900 reads/sec and 1100 writes/sec on 500GB of data would require over 58 spindles to handle the I/O load, or over 8TB allocated. So, yeah, you have more spindles to use, but you need more because the controller has to perform so many operations to do one write to a RAID-DP array! (The same 3900 reads / 1100 writes would only need 39 spindles in RAID10.)
And, snapshots of iSCSI block storage, because they are just files served off the NAS head, require their same size again for a snapshot. Granted, you have a lot of extra space on the array because you had to use so many spindles, but that just means you are wasting a ton of disk just to provide the performance that far fewer RAID10 disks could perform!
And iSCSI itself, though no fault of NetApp's, is not perfect. Certain legal software vendors recommend against it because its 1Gbit storage path speed is considered too slow. The only card I know of that boots off of it, the QLogic 4010, does not support jumbo frames, which causes consternation among networking types as creating headers for 8K SCSI packets adds overhead in calculation time and a reduction in size of the data portion of the packet. (QLogic will fix this with the introduction of the 4050.) And the otherwise excellent VMware ESX Server doesn't support iSCSI until they finally decide to ship ESX 3.0 some time in the next 6 months.
Finally, a flaw of at least the NetApp 900 family is that it only takes Fibre Channel disks. So if you, in a law firm environment, would like high-capacity storage for infrequently accessed data like disk backups and litigation support images, you need a different unit (a FAS3000 or R200) or you need to pay a fortune for 300GB FC drives. This is true as well of the EMC Symmetrix.
So, for these reasons, the NetApp offering is not perfect. Yes, I may unfairly disregard NAS because of a requirement of nutty old legacy apps. And I may misunderstand the performance of RAID-DP. And I haven't surveyed the iSCSI HBA offerings closely enough, so I may be missing an offering that allows both boot-from-SAN and jumbo packets. And I may not be aware of a newer NetApp FAS model that supports both FC and ATA disks. But all of these taken together cause me some dissatisfaction with the NetApp Filer offering.
Stay tuned for my discussion of the EMC product line. I find that they, too, provide no single product that meets all my needs.
This post was not sanctioned by my company (the name of which I've declined to disclose on this blog anyway); all beliefs are my own and do not reflect those of anyone else.
technorati: NetApp NAS SAN iSCSI QLogic EMC
But I've been getting frustrated designing for Network Applications storage as part of my current project. And my prior engagement involved in part carving up storage for Windows servers on an EMC Symmetrix. And I've talked with clients about offerings from Compellent, Falconstor, Hitachi, and HP over the years. I guess I would call my market the middle tier of storage; mainframes are not used and only the occasional Unix system is ever connected to these boxes. Mainly they are for Exchange or Notes clusters, Microsoft SQL server clusters, file clusters and/or NAS, VMware ESX servers, and Windows server boot drives. Ideally the storage controller is used as a mechanism to replicate these systems to another site for DR purposes.
Having worked with some of the major vendors in this middle tier I need to speak my frustration. I do not profess to be a storage engineer so I would not mind any corrections, differences of opinion, or even personal attacks by commenters for this post. I am merely expressing my frustration that nothing is perfect is the small slice of the storage world that I play in. Here are my frustrations:
Network Appliance
This is theoretically the new darling of mid-tier storage, but I grow frustrated with their architecture. Their strength is NAS and in many ways NAS is quite appealing. Consolidating some servers? Want to do it without reconfiguring all your apps? Just have the NAS pretend to be FS1, FS2, NYFS18, whatever. Also, you need far fewer NAS heads than file servers to serve the same amount of data. And, in the NetApp world, snapshots of file systems on NAS are far smaller than snapshots of block storage. And you just have the sense that a custom Linux kernel designed for serving files will do so at a faster rate and far more reliably for far longer than a Windows server.
But, you still need some features that require separate servers when you choose NAS. File share virus protection requires a separate server which must hook into reads and/or writes of every file. Yep, the limiting factor on file serving speeds just became Windows! You could argue that if you protect against viruses via desktop scanning, email gateways, and network edge scanning, you're safe enough, but you still have foreign devices that come onto your network. You could also argue that permissions auditing on a NAS system is not nearly as complete as what you would get in Windows (assuming you set that stuff up). And, in the case of my current client, no NAS can run the docsntss.exe service that DOCS Open requires to properly secure documents. It's a law firm, old document management issue, but if you have that issue, you can't use NAS for your documents. If you were also thinking of using something more sophisticated than RoboCopy or XCOPY /D to replicate data out to branch offices, and you didn't want to put a NetApp NAS in every branch office because you were, I don't know, fiscally responsible or something, then you might want to use FRSv2 in the new R2 release of Windows Server 2003 to replicate file data. Again, that ain't happening with NAS. It's not that I don't still believe in NAS (though you'd have to show me more sophisticated things coming from other vendors, such as running real-time scanning on the NAS head itself) but in my current assignment we've decided against it and in favor of clustered file servers.
However, it is the provision of block storage via iSCSI where I find NetApp falling short. I know that NetApp's strength is that you can make a big-ass RAID group and share all the spindles with a variety of applications. So they're ahead of EMC (Clariion) in my opinion in that they actually can do this. But having RAID4 or RAID-DP be your only options here are wreaking havoc in assigning spindles to write-intensive applications like Exchange and SQL. Maybe I'm gullible or an idiot, but I was given, and am sticking with, a write penalty of 6 for RAID-DP. That means that a group of servers that as a whole are performing 3900 reads/sec and 1100 writes/sec on 500GB of data would require over 58 spindles to handle the I/O load, or over 8TB allocated. So, yeah, you have more spindles to use, but you need more because the controller has to perform so many operations to do one write to a RAID-DP array! (The same 3900 reads / 1100 writes would only need 39 spindles in RAID10.)
And, snapshots of iSCSI block storage, because they are just files served off the NAS head, require their same size again for a snapshot. Granted, you have a lot of extra space on the array because you had to use so many spindles, but that just means you are wasting a ton of disk just to provide the performance that far fewer RAID10 disks could perform!
And iSCSI itself, though no fault of NetApp's, is not perfect. Certain legal software vendors recommend against it because its 1Gbit storage path speed is considered too slow. The only card I know of that boots off of it, the QLogic 4010, does not support jumbo frames, which causes consternation among networking types as creating headers for 8K SCSI packets adds overhead in calculation time and a reduction in size of the data portion of the packet. (QLogic will fix this with the introduction of the 4050.) And the otherwise excellent VMware ESX Server doesn't support iSCSI until they finally decide to ship ESX 3.0 some time in the next 6 months.
Finally, a flaw of at least the NetApp 900 family is that it only takes Fibre Channel disks. So if you, in a law firm environment, would like high-capacity storage for infrequently accessed data like disk backups and litigation support images, you need a different unit (a FAS3000 or R200) or you need to pay a fortune for 300GB FC drives. This is true as well of the EMC Symmetrix.
So, for these reasons, the NetApp offering is not perfect. Yes, I may unfairly disregard NAS because of a requirement of nutty old legacy apps. And I may misunderstand the performance of RAID-DP. And I haven't surveyed the iSCSI HBA offerings closely enough, so I may be missing an offering that allows both boot-from-SAN and jumbo packets. And I may not be aware of a newer NetApp FAS model that supports both FC and ATA disks. But all of these taken together cause me some dissatisfaction with the NetApp Filer offering.
Stay tuned for my discussion of the EMC product line. I find that they, too, provide no single product that meets all my needs.
This post was not sanctioned by my company (the name of which I've declined to disclose on this blog anyway); all beliefs are my own and do not reflect those of anyone else.
technorati: NetApp NAS SAN iSCSI QLogic EMC
Labels: NetApp
Monday, February 20, 2006
as if I didn't post to this blog infrequently enough...
...I now have a new blog based on another interest, making fun of the New York Times Metropolitan Diary column. Read or subscribe if this also interests you.
Monday, February 13, 2006
productivity/GTD blog links
I recently saw a copy of David Allen's book Getting Things Done on my coworker's desk, which I started reading, put down, and recently picked back up again after a four-month hiatus. I may write more later if I see my productivity rising dramatically but I did want to tell my coworker which GTD and productivity-related blogs I read, and figured I could do a little blogrolling in the process...
Gary Slinger: A one-time commenter who writes an excellent blog. I particularly recommend his Implementing GTD with a Blackberry series.
43 Folders: Simple, yet genius productivity tips. If you don't read Merlin already you should.
Tasks and Time Management in Outlook (12): Neat info on task functions in the next version of Outlook
What's the Next Action: Useful tips; I believe the author is part of the TabletPC community
MarkTAW: Updated even less frequently than the blog you are now reading, still contains some excellent GTD articles.
technorati: GTD
Gary Slinger: A one-time commenter who writes an excellent blog. I particularly recommend his Implementing GTD with a Blackberry series.
43 Folders: Simple, yet genius productivity tips. If you don't read Merlin already you should.
Tasks and Time Management in Outlook (12): Neat info on task functions in the next version of Outlook
What's the Next Action: Useful tips; I believe the author is part of the TabletPC community
MarkTAW: Updated even less frequently than the blog you are now reading, still contains some excellent GTD articles.
technorati: GTD
Saturday, February 04, 2006
HDTV upgrade, part 3
I chose to install the version 3.2 of FusionHDTV instead of 3.3Beta1 because I had already seen a bit of instability with the 3.1 drivers (though it may have been due to me switching from cable to antenna and then back to cable without powering off or reconfiguring the software). I first uninstalled the older 3.11 drivers, rebooted, downloaded the 3.2 version, and ran the installer.
During the install, I didn't install the MPEG2 decoders, and said no in response to the curiously worded question "In case of multi card, Would you like to install the additional drivers?" After a restart, I reconnected the FusionHDTV, installed the "FusionHDTV USB, AVStream Capture (ATSC1)" driver version 1.1.0.6, and then fired up the updated FusionHDTV 3.2 program.
Only Over the Air (OTA) HDTV is supported by Windows XP Media Center Edition.
MCE only supports OTA. I guess I remember reading that but then when I was playing with the DViCO on my other PC, I found that I could get DTV channels over the cable and got excited and forgot those words. "Only OTA HDTV is supported by" MCE. So that's entirely why MCE does not recognize the digital channels coming over unencrypted QAM; that's not something it does. It only talks to digital antennas.
D2-1 WCBS-HD: astounding picture, but a little jerky
D4-1 WX-PLUS: Weather Channel, 90% signal; looks okay but is not in the scan function of FusionHDTV for some reason
D4-2 WNBC-HD: slightly worse picture but smoother video than WCBS-HD
D4-4 WNBC4.4: I can't really tell the difference between this and WNBC-HD
D5-1 WNYW DT (Fox): my signal bounces around just below 90%, but it looks okay (a little grainier than the HD stations) and not too jerky
D5-2 WWOR-DT: has sound, but no video despite a 91% signal.
D7-2 WABC+: kind of a grainy signal and jerky video
D7-1 WABC-HD: The signal varies between 75% and 85%, the bottom half of which is below tolerances for a reasonable picture, but I have to say from watching it for 5 minutes the picture looks awesome, not at all jerky.
D9-2 WWOR-DR: 77-90%: The picture started out looking grainy and jerky, but after a minute it is looking pretty clear and good, with a bit of pixelation rather than jerkiness, which I prefer.
D9-2 WNYW-DT (Fox) on 9.2: It's a little washed out and has some compression artifacts where light and shadows meet in the picture
D11-1 WPIX-DT: 78-93% signal, looks very good but the sound was like a tiny fraction off
D13 WNET-SD: 100% signal, don't know if this was the recording of Sesame Street that was on, but the resolution wasn't that great.
D13-3 WNET-13: 100% signal, vaguely grainy picture but could be non-HD source
D13-1 WNET-HD: 100% signal, awesome picture, minimal jerkiness
D31 "i" (turned out to be the same as Pax): 66-79%, no idea what the actual station is since there was an infomercial on, the picture was a bit pixellated but that may be due to the infomercial
D31-2 Pax, D31-3 Worship and D31-4 FaithTV: exactly the same as above
D41-1 WXTV-DT (Univision): 79-95& signal, a bit pixellated but it could have been the "GOLTV" source. No jerkiness.
D51: no signal
D68 WFUT-DT (Univision): 78-90% signal, very clear and non-jerky picture
OK, after that exciting exercise, it was time to bring it into Media Center via the Set Up TV Signal option in Settings-->TV. Media Center recognized the following stations (along with editorial comment; man, I swear this must be my most boring blog entry yet):
1021 WCBS-DT: 6-bar signal in MCE, pretty jerky, with audio that was noticeably out of sync (and eventually turned silent)
1041 WNBC-DT: 5-bar signal, also had audio fairly out of sync (with some skips), and jerky picture
*1051 WNYW DT (Fox): 5-bar signal, picture a bit pixellated but not at all jerky, and no audio problems
*1071 WABC-DT: 5-bar signal, great picture, no jerkiness, no audio problems
1081 WTNH-DT: no signal
1091 WWOR-DT: 4-bar signal, beautiful picture, tolerably jerky, but the audio was way the hell out of sync
1111 WPIX-DT: 5-bar signal, good, slightly jerky picture, audio out of sync and dropping out
1131 WNET-DT: 5-bar signal, really high-res picture, was jerky in MCE Zoom mode 4, but seemed much better in zoom mode 1. Audio was slightly out of sync.
1201 WTXX-DT: no signal
1211 WLIW-DT: no signal
1251 WNYE-DT: no signal
1311 WPXN-DT: 4-bar signal, good picture, audio in sync, unfortunately it's Pax :)
1411 WXTV-DT: 5-bar signal, excellent picture, great audio (in Spanish)
1431 WSAH-DT: no signal
1471 WNJU-DT: no signal
1491 WEDW-DT: no signal
1501 WNJN-DT: no signal
1521 WNJT-DT: no signal
1541 WTBY-DT: no signal
1581 WNJB-DT: no signal
1591 WCTX-DT: no signal
1631 WMBC-DT: no signal
1661 WFME-DT: no signal
1671 WFTY-DT: no signal
1681 WFUT-DT: no signal
This was the signal strength I saw:
During the install, I didn't install the MPEG2 decoders, and said no in response to the curiously worded question "In case of multi card, Would you like to install the additional drivers?" After a restart, I reconnected the FusionHDTV, installed the "FusionHDTV USB, AVStream Capture (ATSC1)" driver version 1.1.0.6, and then fired up the updated FusionHDTV 3.2 program.
Only Over the Air (OTA) HDTV is supported by Windows XP Media Center Edition.
MCE only supports OTA. I guess I remember reading that but then when I was playing with the DViCO on my other PC, I found that I could get DTV channels over the cable and got excited and forgot those words. "Only OTA HDTV is supported by" MCE. So that's entirely why MCE does not recognize the digital channels coming over unencrypted QAM; that's not something it does. It only talks to digital antennas.
D2-1 WCBS-HD: astounding picture, but a little jerky
D4-1 WX-PLUS: Weather Channel, 90% signal; looks okay but is not in the scan function of FusionHDTV for some reason
D4-2 WNBC-HD: slightly worse picture but smoother video than WCBS-HD
D4-4 WNBC4.4: I can't really tell the difference between this and WNBC-HD
D5-1 WNYW DT (Fox): my signal bounces around just below 90%, but it looks okay (a little grainier than the HD stations) and not too jerky
D5-2 WWOR-DT: has sound, but no video despite a 91% signal.
D7-2 WABC+: kind of a grainy signal and jerky video
D7-1 WABC-HD: The signal varies between 75% and 85%, the bottom half of which is below tolerances for a reasonable picture, but I have to say from watching it for 5 minutes the picture looks awesome, not at all jerky.
D9-2 WWOR-DR: 77-90%: The picture started out looking grainy and jerky, but after a minute it is looking pretty clear and good, with a bit of pixelation rather than jerkiness, which I prefer.
D9-2 WNYW-DT (Fox) on 9.2: It's a little washed out and has some compression artifacts where light and shadows meet in the picture
D11-1 WPIX-DT: 78-93% signal, looks very good but the sound was like a tiny fraction off
D13 WNET-SD: 100% signal, don't know if this was the recording of Sesame Street that was on, but the resolution wasn't that great.
D13-3 WNET-13: 100% signal, vaguely grainy picture but could be non-HD source
D13-1 WNET-HD: 100% signal, awesome picture, minimal jerkiness
D31 "i" (turned out to be the same as Pax): 66-79%, no idea what the actual station is since there was an infomercial on, the picture was a bit pixellated but that may be due to the infomercial
D31-2 Pax, D31-3 Worship and D31-4 FaithTV: exactly the same as above
D41-1 WXTV-DT (Univision): 79-95& signal, a bit pixellated but it could have been the "GOLTV" source. No jerkiness.
D51: no signal
D68 WFUT-DT (Univision): 78-90% signal, very clear and non-jerky picture
OK, after that exciting exercise, it was time to bring it into Media Center via the Set Up TV Signal option in Settings-->TV. Media Center recognized the following stations (along with editorial comment; man, I swear this must be my most boring blog entry yet):
1021 WCBS-DT: 6-bar signal in MCE, pretty jerky, with audio that was noticeably out of sync (and eventually turned silent)
1041 WNBC-DT: 5-bar signal, also had audio fairly out of sync (with some skips), and jerky picture
*1051 WNYW DT (Fox): 5-bar signal, picture a bit pixellated but not at all jerky, and no audio problems
*1071 WABC-DT: 5-bar signal, great picture, no jerkiness, no audio problems
1081 WTNH-DT: no signal
1091 WWOR-DT: 4-bar signal, beautiful picture, tolerably jerky, but the audio was way the hell out of sync
1111 WPIX-DT: 5-bar signal, good, slightly jerky picture, audio out of sync and dropping out
1131 WNET-DT: 5-bar signal, really high-res picture, was jerky in MCE Zoom mode 4, but seemed much better in zoom mode 1. Audio was slightly out of sync.
1201 WTXX-DT: no signal
1211 WLIW-DT: no signal
1251 WNYE-DT: no signal
1311 WPXN-DT: 4-bar signal, good picture, audio in sync, unfortunately it's Pax :)
1411 WXTV-DT: 5-bar signal, excellent picture, great audio (in Spanish)
1431 WSAH-DT: no signal
1471 WNJU-DT: no signal
1491 WEDW-DT: no signal
1501 WNJN-DT: no signal
1521 WNJT-DT: no signal
1541 WTBY-DT: no signal
1581 WNJB-DT: no signal
1591 WCTX-DT: no signal
1631 WMBC-DT: no signal
1661 WFME-DT: no signal
1671 WFTY-DT: no signal
1681 WFUT-DT: no signal
This was the signal strength I saw:
Labels: Media Center
HDTV upgrade, part 2
I am writing this on my TabletPC as I sit dejectedly in front of my TV and Media Center PC, trying to get my new DViCO FusionHDTV5 USB to work. Sorry for all the drama but it's my blog, if you don't like it, you are welcome to go read anti-"give our abilities time" blogs on myspace that are primarily about pretty ponies if you don't like it.
First, the DViCO. I bought it after reading some good reviews in a few places. I needed a USB HDTV tuner because my HP z552 came with its 3 slots filled with two tuner cards and a wireless NIC. It was pretty cheap ($150), seemed aesthetically pleasing, and the manufacturer seemed to be a leader in the field. I bought it from SnapStream as recommended since they threw in an antenna (the included one was reportedly crap). It showed up Thursday and I had an hour to play with it on my regular home PC.
To my suprise and pleasure, I was able to get unencrypted digital channels through the cable line which looked much better than their equivalent analog cable signal. I was psyched to hook it up to the Media Center PC and start doing digital DVR.
Last might, after an evening out with friends, I spent a few hours trying to get the DViCO working on MCE. I installed the software, skipping the included MPEG2 decoder, and connected the tuner as shown in this awesome TabletPC drawing:
After scanning the CATV channels, the FusionHDTV software worked as it had on my PC. However, Media Center refused to see any signal on the tuner.
After trying with also with the antenna and then giving up in disgust last night, today's troubleshooting leads me to believe that MCE only thinks of the tuner as a "digital antenna", so it is only offering me the OTA channels it thinks I should get given my ZIP code. I believe this because no matter how it scans, Media Center TV Tuner Set-up only shows signals that seem to be OTA:
And of course that leads to this message appearing on every new channel:
I have downloaded updated software from the DViCO web site and will continue to report on my progress...
Update: My problems are because MCE only supports OTA, not clear QAM. See part 3 for more info...
--sbreck
HDTV upgrade, part 1 (about the LCD I bought and my (successful) attempt to get it to display widescreen 720p images)
technorati: Media Center HDTV
Labels: Media Center
