Discussion:
What filesystem?
(too old to reply)
John Bass
2006-03-03 21:11:55 UTC
Permalink
Hi,

I hope everyone doesn't mind if I repeat a question. One of the few responses I did get said I should be more complete in asking my question.

My boss is trying to choose one distributed file system from NFS and MS-DFS and so on, from my feeling, AFS sounds like the best choice, it is secure and is being used by many large institutes, such as Harvard, MIT, NASA, Stanford, etc. Actually GrandCentral.org says they use it, I cant tell for sure if that means the whole place, or one department, or a couple of people in a lab.

In order to convince my boss, I have searched via Internet, expected to find some kind of comparison between AFS, NFS and MS-DFS, or some web pages that could have something tracking the usage of each file system, which can be used to prove/indicate AFS is better than the others. But except for an outdated IBM AFS whitepaper, I didn't find any solid evidence or data that I can present to my boss. I had hoped to find a source of information such as websites that track Linux installs or MSIE/Opera/Mozilla usage (or this), but it seems hards to find. I see openafs.org has "Success Stories", but there are only five!

Anybody can see that some of very smart people made and use AFS, but how can an overworked person like me convince my also overworked boss that we should spend the money and time necessary to learn, deploy, train, and support AFS enterprize-wide?

I wonder if anyone of you have had this kind of situation before, or may have something that I can use to convince my boss.

Thank you in advance for any kindly assistance.


John Bass


__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
ted creedon
2006-03-03 21:29:19 UTC
Permalink
1. What's your line of business?
2. Do Road Warriors need secure access?
3. What OS's are in use?
4. Any federal regulations involved?
5. How many users?
6. How many sites?
7. Do you need byte level locking?
8. Is Morgan Stanley big enough?



http://www.pmw.org/afsbpw05/workshop.html look at the NW Air ppt slides. AFS
is accepted by the FAA.



tedc



_____

From: openafs-info-***@openafs.org [mailto:openafs-info-***@openafs.org]
On Behalf Of John Bass
Sent: Friday, March 03, 2006 1:12 PM
To: openafs-***@openafs.org
Subject: [OpenAFS] What filesystem?




Hi,

I hope everyone doesn't mind if I repeat a question. One of the few
responses I did get said I should be more complete in asking my question.

My boss is trying to choose one distributed file system from NFS and MS-DFS
and so on, from my feeling, AFS sounds like the best choice, it is secure
and is being used by many large institutes, such as Harvard, MIT, NASA,
Stanford, etc. Actually GrandCentral.org says they use it, I cant tell for
sure if that means the whole place, or one department, or a couple of people
in a lab.

In order to convince my boss, I have searched via Internet, expected to find
some kind of comparison between AFS, NFS and MS-DFS, or some web pages that
could have something tracking the usage of each file system, which can be
used to prove/indicate AFS is better than the others. But except for an
outdated IBM AFS whitepaper, I didn't find any solid evidence or data that I
can present to my boss. I ha d hoped to find a source of information such
as websites that track Linux <http://counter.li.org/> installs or
MSIE/Opera/Mozilla
<http://www.websidestory.com/products/web-analytics/datainsights/spotlight/0
5-10-2005.html> usage (or this
<http://google.blognewschannel.com/index.php/archives/2005/05/25/firefox-num
ber-one-here/> ), but it seems hards to find. I see openafs.org has
"Success Stories", but there are only five!

Anybody can see that some of very smart people made and use AFS, but how can
an overworked person like me convince my also overworked boss that we should
spend the money and time necessary to learn, deploy, train, and support AFS
enterprize-wide?

I wonder if anyone of you have had this kind of situation before, or may
have something that I can use to convince my boss.

Thank you in advance for any kindly assistance.


John Bass

__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
Hendrik Hoeth
2006-03-03 22:06:56 UTC
Permalink
Post by ted creedon
http://www.pmw.org/afsbpw05/workshop.html look at the NW Air ppt
slides. AFS is accepted by the FAA.
actually it is United, not NW. The title is "AFS Cell Profile" and the
talk was held by Dexter Kimball. Now you should be able to find it ;-)
--
Fashion is a form of ugliness so intolerable that we have to alter
it every six months. -- Oscar Wilde
Horst Birthelmer
2006-03-03 22:22:55 UTC
Permalink
Post by John Bass
Hi,
I hope everyone doesn't mind if I repeat a question. One of the
few responses I did get said I should be more complete in asking my
question.
My boss is trying to choose one distributed file system from NFS
and MS-DFS and so on, from my feeling, AFS sounds like the best
choice, it is secure and is being used by many large institutes,
such as Harvard, MIT, NASA, Stanford, etc. Actually
GrandCentral.org says they use it, I cant tell for sure if that
means the whole place, or one department, or a couple of people in
a lab.
A comparison between those three file systems would be highly unfair.
First, because we (the people on the list) wouldn't be that
objective ;-) and second, because they're very different and neither
of them will solve all of your file providing problems in your
organization.
What features you're defining as important and which you'll never
going to need is up to you.

Keep in mind - no matter how you and/or your boss will decide - that
AFS is a _distributed_ file system.
It does start to make sense if you have more file servers.
It is highly scalable from one fileserver (which is kinda trivial) to
a complete distributed cell with servers placed all over the world.
You can choose any level of distribution in between.

...
Post by John Bass
Anybody can see that some of very smart people made and use AFS,
but how can an overworked person like me convince my also
overworked boss that we should spend the money and time necessary
to learn, deploy, train, and support AFS enterprize-wide?
This is something for you to decide...
It will definitely cost you some money and time.

Horst
Volker Lendecke
2006-03-03 22:30:04 UTC
Permalink
Post by Horst Birthelmer
Post by John Bass
Anybody can see that some of very smart people made and use AFS,
but how can an overworked person like me convince my also
overworked boss that we should spend the money and time necessary
to learn, deploy, train, and support AFS enterprize-wide?
This is something for you to decide...
It will definitely cost you some money and time.
If you have one or two servers, AFS probably is not worth
the hassle. But AFS really pays off when you run out of
fingers to count your servers. I see the initial cost in
particular when you're new to AFS as relatively high, but in
the long run with a lot of servers around the world, you
will start to love it.

Volker
Samba Team ;-)
Horst Birthelmer
2006-03-03 22:35:17 UTC
Permalink
Post by Volker Lendecke
Post by Horst Birthelmer
Post by John Bass
Anybody can see that some of very smart people made and use AFS,
but how can an overworked person like me convince my also
overworked boss that we should spend the money and time necessary
to learn, deploy, train, and support AFS enterprize-wide?
This is something for you to decide...
It will definitely cost you some money and time.
If you have one or two servers, AFS probably is not worth
the hassle. But AFS really pays off when you run out of
fingers to count your servers. I see the initial cost in
particular when you're new to AFS as relatively high, but in
the long run with a lot of servers around the world, you
will start to love it.
Volker
Samba Team ;-)
Now that's nice ...

I just wanted to add, that if you reach that size Volker was talking
about, you also ran out of options.
AFS is the only file system being able to help you in that case.

Horst
Christopher D. Clausen
2006-03-04 10:46:12 UTC
Permalink
Post by Horst Birthelmer
Post by Volker Lendecke
If you have one or two servers, AFS probably is not worth
the hassle. But AFS really pays off when you run out of
fingers to count your servers. I see the initial cost in
particular when you're new to AFS as relatively high, but in
the long run with a lot of servers around the world, you
will start to love it.
I just wanted to add, that if you reach that size Volker was talking
about, you also ran out of options.
AFS is the only file system being able to help you in that case.
Microsoft's Dfs should scale as well. Of course, it essentially only
works on Windows, but if you are mostly a Windows shop, it may be the
best way to go. I am a particular fan of the multi-master read-write
replication possibilities it offers, something that is not currently
possible with OpenAFS. And MS Dfs is suposed to get even better in
Windows 2003 R2.

Some will argue that the "last write wins" algorithm it uses isn't a
good idea, but for user home directories I think it is the best thing to
do and possibly the only solution out there for Windows.

If there are further questions, I'd suggest discussions on the #openafs
IRC channel on the freenode network.

<<CDC
--
Christopher D. Clausen
***@UIUC SysAdmin
Horst Birthelmer
2006-03-04 16:00:44 UTC
Permalink
Post by Christopher D. Clausen
Post by Horst Birthelmer
Post by Volker Lendecke
If you have one or two servers, AFS probably is not worth
the hassle. But AFS really pays off when you run out of
fingers to count your servers. I see the initial cost in
particular when you're new to AFS as relatively high, but in
the long run with a lot of servers around the world, you
will start to love it.
I just wanted to add, that if you reach that size Volker was talking
about, you also ran out of options.
AFS is the only file system being able to help you in that case.
Microsoft's Dfs should scale as well. Of course, it essentially
only works on Windows, but if you are mostly a Windows shop, it may
be the best way to go. I am a particular fan of the multi-master
read-write replication possibilities it offers, something that is
not currently possible with OpenAFS. And MS Dfs is suposed to get
even better in Windows 2003 R2.
My experience with DFS is very limited (I actually didn't have more
than a few contacts with it), but is there any possibility for an
administrator to move any of the shared data transparently to a new
location without all the clients to notice?
Is there any possibility to manage the 'shares' at all?

I'm a little confused how and why that's a match for AFS. We were
talking about a large worldwide installation with possible
mountpoints from different cells etc.

I quickread some of the information from Microsoft on this topic but
didn't find any information about any improvements of DFS beyond that
'name service' for windows fileservers. (well, it's over simplified,
I know)
Post by Christopher D. Clausen
Some will argue that the "last write wins" algorithm it uses isn't
a good idea, but for user home directories I think it is the best
thing to do and possibly the only solution out there for Windows.
Those algorithms weren't point here ... ;-)
Post by Christopher D. Clausen
If there are further questions, I'd suggest discussions on the
#openafs IRC channel on the freenode network.
I know about the channel. I'm in there, too, most of the time, but
not that active ;-) Sorry.


Horst
Christopher D. Clausen
2006-03-04 16:45:21 UTC
Permalink
Post by Horst Birthelmer
Post by Christopher D. Clausen
Post by Horst Birthelmer
Post by Volker Lendecke
If you have one or two servers, AFS probably is not worth
the hassle. But AFS really pays off when you run out of
fingers to count your servers. I see the initial cost in
particular when you're new to AFS as relatively high, but in
the long run with a lot of servers around the world, you
will start to love it.
I just wanted to add, that if you reach that size Volker was talking
about, you also ran out of options.
AFS is the only file system being able to help you in that case.
Microsoft's Dfs should scale as well. Of course, it essentially
only works on Windows, but if you are mostly a Windows shop, it may
be the best way to go. I am a particular fan of the multi-master
read-write replication possibilities it offers, something that is
not currently possible with OpenAFS. And MS Dfs is suposed to get
even better in Windows 2003 R2.
My experience with DFS is very limited (I actually didn't have more
than a few contacts with it), but is there any possibility for an
administrator to move any of the shared data transparently to a new
location without all the clients to notice?
Is there any possibility to manage the 'shares' at all?
You can transparently move a share by setting up a new share on the new
fileserver, create a replica on it, wait for replication to finish, and
delete the old share. More work is involved than a simple vos move, but
I would strongly prefer read-write realtime replicas to the simplicity
of vos move and would likely have everything setup with a least one
mirror anyway and then there wouldn't be as much of a need to move
things. The server could just go down while rebooting for patches,
hardware upgrades, etc.

Depends on what you mean by manage. There is a nice Dfs management GUI
MMC plugin. If you want to manage everything from the command line or
from non-Windows you will likely be in some pain. And you are correct
that its not quite as nice as AFS b/c one is not always an admin on all
the local servers and thus may not be able to just create shares on
whatever random server is in the Dfs name space. Domain Admins need to
be granted local administrator rights on each machine to create the
shares or otherwise have them created by the local admins.
Post by Horst Birthelmer
I'm a little confused how and why that's a match for AFS. We were
talking about a large worldwide installation with possible
mountpoints from different cells etc.
Dfs reparse points are links to various Windows shares. (They are
similar to junctions within NTFS.) Its different from AFS in that the
servers don't need to share a common KeyFile and can utilize the same
UNC namespace. (Similar to allowing multiple groups to run their own
cells, without the need to tokens and PTS entries in each seperate
cell.)

You could argue that NFS with the proper auto-mount mappings in NIS and
sometype of file replication could provide infrastructure similar to
Dfs.
Post by Horst Birthelmer
I quickread some of the information from Microsoft on this topic but
didn't find any information about any improvements of DFS beyond that
'name service' for windows fileservers. (well, it's over simplified,
I know)
Actually, that is pretty much all it is. Its a namespace for CIFS
shares. The replicas work with the File Replication Service (frs) and
can operate even without a domain controller, although most places will
want the data hosted in active directory. The file replication can be
more advanced than one to many replication. Active Directory has a
concept of "sites" and this can be utilized to save data transmission
time over WAN links between branch offices. Group Policy can be used to
point different sets of machines at different servers by default,
failing over to other sites when needed.

MS Dfs (not to confuse it with the OpenGroup's DCE DFS stuff) works
differently from how AFS works, with a different view of where things
should be managed. IMHO, AFS is limited b/c different cells tie PTS
entries to specific Kerberos credentials and one needs to seperate
authenticate to each cell, not each server. Dfs uses only Kerberos
tickets to logon to each server and provides failover if one of them is
down. The servers can be individually managed by seperate entities as
long as they are joined to the same AD forest. Adding a server to a Dfs
namespace does not imply sharing admin access to all other server admins
like how AFS works with its shared key. (Needing to be in the UserList
on all DB servers to add volumes and vldb entries is a serious
limitation and probably prevents some organizations from adopting AFS.)

Its not a solution for everyone, but its an option. Its more integrated
with Microsoft technologies than is usually liked in the organizations
that use AFS, but IMHO it provides a more useful set of features,
especially for Microsoft-only organizations.

<<CDC
Jeffrey Altman
2006-03-04 17:30:58 UTC
Permalink
Post by Christopher D. Clausen
Actually, that is pretty much all it is. Its a namespace for CIFS
shares. The replicas work with the File Replication Service (frs) and
can operate even without a domain controller, although most places will
want the data hosted in active directory. The file replication can be
more advanced than one to many replication. Active Directory has a
concept of "sites" and this can be utilized to save data transmission
time over WAN links between branch offices. Group Policy can be used to
point different sets of machines at different servers by default,
failing over to other sites when needed.
With Windows 2003 R2, incremental propagation of replicated data for
files and active directory entries can be performed either in real time
or in batch modes. The data exchange can either use socket connections
or message based transports such as e-mail. This is a major boon for
branch offices that have only a small number of machines or are in
locations without dedicated high speed networks. The model is last
write wins. This is sufficient for many application models.

Note that this model makes it impossible to use file locking to protect
files from overwrites. This is a conscious decision that is made by the
administrator when a share is replicated.

Jeffrey Altman
Rodney M Dyer
2006-03-04 18:35:28 UTC
Permalink
Post by Christopher D. Clausen
Actually, that is pretty much all it is. Its a namespace for CIFS
shares. The replicas work with the File Replication Service (frs) and can
operate even without a domain controller, although most places will want
the data hosted in active directory. The file replication can be more
advanced than one to many replication. Active Directory has a concept of
"sites" and this can be utilized to save data transmission time over WAN
links between branch offices. Group Policy can be used to point different
sets of machines at different servers by default, failing over to other
sites when needed.
And for these reasons you can forget common name space ACLs, groups, and
such unless you have a single AD where all your shares are from AD domain
member servers and you are a member of the domain. And since it is CIFS
based, there's no concept of a local cache to reduce network traffic. As
well the Dfs root node must be authenticated to, meaning clients at home
(or remote on the Internet) must be members of the AD domain that the root
node belongs to. And btw, who in their right mind would share out CIFS
over the Internet these days? A strict comparison of AFS to Dfs would look
something like...

AFS: Yes, Yes, Yes, Yes, Yes, Yes
Dfs: No, No, Sometimes, Maybe, Unsure, Not documented, etc. etc...

About the only thing Dfs has going for it -are- the automatic replication
facilities, which most people in the AFS world find rather odd. You see,
in the AFS world, most of the tree is in the form of read-only replicated
sites. Only the user volumes and a few others are read-write. And when a
read-write volume goes down, you really don't want to "fail-over". The
business of read-write volumes for critical data should be in the form of
RAID, not fail-over volumes.

It's kind of hard to explain, but Windows people and Unix people think
differently about what they are serving out. AFS grew up in a world where
Unix admins wanted to distribute executable "applications" as well as data
from a single name space tree. In the enterprise, Unix workstations
actually run their applications off of the network. Very few people using
AFS actually install applications on their local workstations. This makes
application distribution to thousands of workstations into a single vos
replicate command. Try that with Windows! Windows application
installation and distribution is a royal pain in the ass for admins and is
why Windows has such a high cost of ownership for the enterprise. Ever try
running applications off your Windows server Dfs tree? For that matter,
how many Windows applications have you come across that are actually made
to install on the network? AFS has the concept of the "dotted" cell path
which gets you to the read-write tree of volumes. At our site we release
an application and test it using the "dotted" path before any of our real
workstations see it in the "read-only" tree. When it is tested and ready,
we issue a vos release command, which updates all the read-only sites and
makes the application available to all the workstations. Through blood,
sweat, and tears, we've rigged our Windows XP environment to work much like
the way our Unix environment works and it has saved us a hell of a lot of
admin time.

<on soapbox>My conspiracy theory is that in the Microsoft Windows world of
the 90's, there was virtually no incentive for Microsoft to "help" vendors,
or establish rigorous rules, methods, and policies for how applications can
run off the network. Applications run faster off of the local disk, and
Microsoft had no intention of helping vendors like Novell with applications
that can run off their network. For sure, even Microsoft didn't want their
Office suite of applications to "appear" to run slowly compared to the Unix
world, where the apps typically were executed off of NFS shares. It
certainly was a big boon for hard drive makers since users keep needing
bigger and bigger drives to store their applications on. Remember how slow
the network used to be?<off soapbox>

Windows admins on the other hand tend to put DATA, not applications, on the
network. Most Dfs trees lead only to user volumes and some shared database
files, or common Office shares. The auto-replication facilities of DFS
would work fine in this circumstance.

We've setup a Dfs tree along side our AFS tree, but haven't used it much at
all. Our primary intention was to have something to go to if the OpenAFS
Windows client turned out to be unsupported when Transarc was broken
up. Companies like Network Appliance sell NAS gear that can be used in Dfs
environments too. Now, the OpenAFS Windows client has great support
through developers like Jeffrey Altman of Secure-Endpoints.

Frankly, Microsoft's support of Dfs so far has been abysmal. They really
don't promote it at all. They see it as just something that some
enterprises might be interested in doing. And only those enterprises that
can afford to get enterprise level support contracts get any real help. In
effect, Microsoft is using the enterprise for a test-bed of technologies
like Dfs. It's only when they actually get enough documentation and
features into it that they start to promote it to the world. I mean three
years ago, you were lucky to find anything beyond some obscure white papers
on Dfs. Dfs does work, but in the real world when most Windows IT groups
are interested in setting up big trees of data they actually end up turning
to NAS or SAN.

Like you said, Dfs is primarily an all Windows technology anyway. If you
need anything heterogeneous then you must turn to AFS.

Rodney

Rodney M. Dyer
Windows Systems Programmer
Mosaic Computing Group
William States Lee College of Engineering
University of North Carolina at Charlotte
Email: ***@uncc.edu
Web: http://www.coe.uncc.edu/~rmdyer
Phone: (704)687-3518
Help Desk Line: (704)687-3150
FAX: (704)687-2352
Office: Cameron Applied Research Center, Room 232
Christopher D. Clausen
2006-03-06 15:17:23 UTC
Permalink
Post by Rodney M Dyer
Post by Christopher D. Clausen
Actually, that is pretty much all it is. Its a namespace for CIFS
shares. The replicas work with the File Replication Service (frs)
and can operate even without a domain controller, although most
places will want the data hosted in active directory. The file
replication can be more advanced than one to many replication. Active
Directory has a concept of "sites" and this can be utilized
to save data transmission time over WAN links between branch
offices. Group Policy can be used to point different sets of
machines at different servers by default, failing over to other
sites when needed.
And for these reasons you can forget common name space ACLs, groups,
and such unless you have a single AD where all your shares are from
AD domain member servers and you are a member of the domain. And
since it is CIFS based, there's no concept of a local cache to reduce
network traffic. As well the Dfs root node must be authenticated to,
meaning clients at home (or remote on the Internet) must be members
of the AD domain that the root node belongs to. And btw, who in
their right mind would share out CIFS over the Internet these days? A
strict comparison of AFS to Dfs would look something like...
Well, at UIUC we have a common campus-wide domain so its a FEATURE that
all shares are under it. Seamless Kerberos logons between disparately
managed file servers. I don't think its possible to delegate control
like that with AFS.

I'm going to have to disagree with your statement of caching as well,
unless I totally misunderstand how offline files work. And, that is in
fact another feature, offline access to remote files. (AFS sort of has
this on Windows, but only on Windows and using the same methods.)

As to sharing CIFS over the Internet, I certainly would do so but
unfortunately the matter has been decided and the campus firewall blocks
MS RPC and CIFS traffic. However, the common refrain I keep hearing is
"use the VPN client" to gain access to campus resources. I am not
really a fan of the Cisco VPN that UIUC offers, but there is nothing
preventing me from running my own "Routing and Remote Access" server
allowing access with the much friendlier native Windows client. (One
could argue that VPNed access provides a higher level of encryption than
AFS does, although one can also use the VPN with AFS.)
Post by Rodney M Dyer
AFS: Yes, Yes, Yes, Yes, Yes, Yes
Dfs: No, No, Sometimes, Maybe, Unsure, Not documented, etc. etc...
Again, depends on your needs. After working with NTFS ACLs I find AFS
ones limiting. There appears to be no way to allow one to create mount
points and not change ACLs. Why is that? There isn't a easy way to
have shared scratch space, preventing others from nuking not their own
files. There is no way to grant access to directory and NOT have that
same access get inherited to newly created sub-directories.

Not needed to setup the equivalent of PTS is a feature IMHO as well.
Why do I need an account to the filesystem? Or for that matter, a
token. Not needed to have end-users worry about tokens can be a
feature. (Not being able to deny access to the filesystem by unloging
is a non-feature.)

I'm going to have to say that AFS has some strange "undocumented
features" as well. Like how the owner of the root of a volume gets
extra rights... And the system:ptsviewers group... and the refresh time
on IP-based ACLs... (I could have missed all this in the pages of
documentation though, and yes, I know its being worked on.)
Post by Rodney M Dyer
About the only thing Dfs has going for it -are- the automatic
replication facilities, which most people in the AFS world find
rather odd. You see, in the AFS world, most of the tree is in the
form of read-only replicated sites. Only the user volumes and a few
others are read-write. And when a read-write volume goes down, you
really don't want to "fail-over". The business of read-write volumes
for critical data should be in the form of RAID, not fail-over
volumes.
Well, not everyone here at UIUC can afford RAID setups. I've worked at
several departments where the file-server is just a desktop machine with
a single hard drive. Convincing them to spend money on an entire
redundant server is easier than on expensive disk. ("I got this 300GB
USB drive for $200. Why do you need $7000 for an array?") We both know
that RAID is good, and if you have the money use RAID with a
replicatation partner.

RAID doesn't help if a processor or a power supply goes bad, or on OS
upgrade freaks out or whatever. As in, its not protection against
SERVER downtime. It protection against only disk failures. (And being
in a place that uses 3 year old Dell desktops as fileservers, this is a
concern.)

Only the user volumes are read-write? All of my important data is in
user volumes. Read-only data is easy to keep online, just replicate it.
The important stuff is what people are working on RIGHT NOW and if its
down bad things happen.
Post by Rodney M Dyer
It's kind of hard to explain, but Windows people and Unix people think
differently about what they are serving out. AFS grew up in a world
where Unix admins wanted to distribute executable "applications" as
well as data from a single name space tree. In the enterprise, Unix
workstations actually run their applications off of the network.
Warning: I am a Windows person.

I'm not at an enterprise. I'm at a University. My systems at home are
more advanced than most of the "production" services I find in use
throughout campus. (Not at the campus IT level, but at a department
level: single fileserver, 100 clients max.)
Post by Rodney M Dyer
Very few people using AFS actually install applications on their
local workstations. This makes application distribution to thousands
of workstations into a single vos replicate command. Try that with
Windows! Windows application installation and distribution is a
royal pain in the ass for admins and is why Windows has such a high
cost of ownership for the enterprise. Ever try running applications
off your Windows server Dfs tree?
Yes, actually. Once you trick them to install it works just fine.
Post by Rodney M Dyer
For that matter, how many Windows
applications have you come across that are actually made to install
on the network?
Yes, this was annoying. Had to create local partition "S:" (for
software) and install things there, then copy into AFS and mount the
path as S: on each client machine. Same trick for Dfs installs as well.
Almost everything works using this technique. The problem appears to be
mainly the installers, not the apps themselves. Got to watch things
that install as system services or explorer extensions though. That
causes PAIN. Unless of course you ACL everything system:anyuser rl so
that the system can read it. Probably not legal to do in all
instances... but that's another matter...

Another problem is 100BASE networks are at least 4 times slower than
hard drives. Who wants to wait for that? And, I have 8GBs of software
installed on my Windows machines. I believe the Windows AFS cache size
limit is about 1.3GBs. Say all you want about bloated software, but its
a fact of life.

Do you really run Visual Studio and AutoCAD directly out of AFS?
Post by Rodney M Dyer
AFS has the concept of the "dotted" cell path which
gets you to the read-write tree of volumes. At our site we release
an application and test it using the "dotted" path before any of our
real workstations see it in the "read-only" tree. When it is tested
and ready, we issue a vos release command, which updates all the
read-only sites and makes the application available to all the
workstations. Through blood, sweat, and tears, we've rigged our
Windows XP environment to work much like the way our Unix environment
works and it has saved us a hell of a lot of admin time.
I only have about 20 machines that I directly maintain. Again, most of
the data is in user volumes that people access from their own computers.

And the vos release stuff is annoying as well. Why should I need to
write an app to handle allowing users to replicate volumes? What about
the user education issue? "Oh the website isn't up to date b/c no one
vos released it yet."

Yes, having separate RO and RW paths is good in some cases. Sometimes
it isn't.
Post by Rodney M Dyer
<on soapbox>My conspiracy theory is that in the Microsoft Windows
[snip]
Remember how slow the network used to be?<off soapbox>
Oh I agree. Your view of things and some other view may be different.
I'm just supplying the reasons why someone may choose MS Dfs over
OpenAFS. (Having already done the research and picked AFS.)
Post by Rodney M Dyer
Windows admins on the other hand tend to put DATA, not applications,
on the network. Most Dfs trees lead only to user volumes and some
shared database files, or common Office shares. The auto-replication
facilities of DFS would work fine in this circumstance.
Being a Windows admin, I concur with that. Why waste the fileserver
serving out the same exact app everyday? Why not utilize it for files
that people actually need to share? (I'm on a 10BASE network right now.
I can assure you that there would be much time wasted waiting for Office
to startup out of a network filesystem.)

Also, I run some Debian and Mac OS machines as well. Its still easier
to install things locally. Only Solaris appears to actually encourage
remotely mountable applications.
Post by Rodney M Dyer
We've setup a Dfs tree along side our AFS tree, but haven't used it
much at all. Our primary intention was to have something to go to if
the OpenAFS Windows client turned out to be unsupported when Transarc
was broken up. Companies like Network Appliance sell NAS gear that
can be used
in Dfs environments too. Now, the OpenAFS Windows client has great
support through developers like Jeffrey Altman of Secure-Endpoints.
Well, we have no money for support of any kind and for us both Microsoft
products and OpenAFS are free so its easy to try and compare.
Post by Rodney M Dyer
most Windows IT groups are
interested in setting up big trees of data they actually end up
turning to NAS or SAN.
Most departments here at UIUC can easily afford $2000 for an additional
fileserver for replication but cannot afford SAN gear, even low-cost
iSCSI stuff. In the end, it comes down to what you can do with the
money you have. Dfs is still better than just plain CIFS on a single
fileserver.
Post by Rodney M Dyer
Like you said, Dfs is primarily an all Windows technology anyway. If
you need anything heterogeneous then you must turn to AFS.
This is entirely true. Again, some places are all Windows where Dfs
would make sense. I believe this is why you see more universities using
AFS, they tend to allow more freedom in IT. Corporations pick a
standard and that's it.

<<CDC
--
Christopher D. Clausen
***@UIUC SysAdmin
Jeffrey Altman
2006-03-06 22:18:19 UTC
Permalink
Post by Christopher D. Clausen
Well, at UIUC we have a common campus-wide domain so its a FEATURE that
all shares are under it. Seamless Kerberos logons between disparately
managed file servers. I don't think its possible to delegate control
like that with AFS.
I believe that Rodney's point is that in most academic institutions one
can hardly assume that all student and faculty computers are members of
the domain. At many schools there is a central Kerberos and Active
Directory infrastructure, but there are also many departmental Active
Directories that are not in any way connected to the centrally managed ones.

What you are describing as seemless Kerberos logons is the ability of
the Windows CIFS client to determine which server the user is attempting
to contact and to obtain a Kerberos service ticket for the CIFS service
on that server when the user has already obtained a TGT at logon.

This single sign-on capability is not provided by OpenAFS for Windows.
However, the new NetIDMgr with its AFS plug-in takes away much of the
inconvenience of supporting multiple cells. One Kerberos realm (or
Active Directory domain) can be used as the authentication source for
multiple AFS cells. Therefore, it is possible to give administrative
control over cells to different groups within the organization while
requiring the end users to perform only a single initial authentication.

For example, the ATHENA.MIT.EDU realm provides authentication to the
following cells in the public CellServDB: athena.mit.edu, dev.mit.edu,
net.mit.edu, sipb.mit.edu, and soap.mit.edu. Using cross-realm
authentication the ATHENA.MIT.EDU principal can also be used to obtain
tokens for many other cells including andrew.cmu.edu, iastate.edu, etc.

With NetIDMgr and its AFS plug-in, the user can configure a list of
cells for which AFS tokens are to be obtained whenever the associated
Kerberos TGT is acquired.

NetIDMgr will replace afscreds.exe in a future release of OpenAFS
for Windows.
Post by Christopher D. Clausen
I'm going to have to disagree with your statement of caching as well,
unless I totally misunderstand how offline files work. And, that is in
fact another feature, offline access to remote files. (AFS sort of has
this on Windows, but only on Windows and using the same methods.)
The CIFS offline files mechanism is not exactly caching. It provides
a mechanism by which the contents of a remote share can be locally
replicated at logon and/or logoff or upon user request similar to how
roaming profiles work. This is a valuable functionality and I would
not claim that the AFS client on Windows has it. The only reason it
would work with AFS on Windows is that the method of accessing AFS on
Windows is via the CIFS client which implements the "Client Side
Caching" functionality.
Post by Christopher D. Clausen
As to sharing CIFS over the Internet, I certainly would do so but
unfortunately the matter has been decided and the campus firewall blocks
MS RPC and CIFS traffic. However, the common refrain I keep hearing is
"use the VPN client" to gain access to campus resources. I am not
really a fan of the Cisco VPN that UIUC offers, but there is nothing
preventing me from running my own "Routing and Remote Access" server
allowing access with the much friendlier native Windows client. (One
could argue that VPNed access provides a higher level of encryption than
AFS does, although one can also use the VPN with AFS.)
Blocking the ports at the organizational boundary and using VPNs is the
recommendation of Microsoft.
Post by Christopher D. Clausen
Post by Rodney M Dyer
AFS: Yes, Yes, Yes, Yes, Yes, Yes
Dfs: No, No, Sometimes, Maybe, Unsure, Not documented, etc. etc...
Again, depends on your needs. After working with NTFS ACLs I find AFS
ones limiting. There appears to be no way to allow one to create mount
points and not change ACLs.
This is conscious decision to prevent random people with write privs
from being able to add a link to other space that has different ACL
requirements. The notion is that adding a link to different ACL space
would be the same as making ACL changes.
Post by Christopher D. Clausen
Why is that? There isn't a easy way to
have shared scratch space, preventing others from nuking not their own
files. There is no way to grant access to directory and NOT have that
same access get inherited to newly created sub-directories.
In Windows, the concept of inheritance was added after the fact. In
general, if you are locking down a directory, you want it to be locked
down. If you want to give users individual space, an administrator
should create the directory and set the appropriate ACLs on it.
Post by Christopher D. Clausen
Not needed to setup the equivalent of PTS is a feature IMHO as well. Why
do I need an account to the filesystem? Or for that matter, a token.
Not needed to have end-users worry about tokens can be a feature. (Not
being able to deny access to the filesystem by unloging is a non-feature.)
In other words, you would like to be able to use Windows SIDs as AFS IDs
and have a PTS RPC service as a front end to active directory.
Post by Christopher D. Clausen
I'm going to have to say that AFS has some strange "undocumented
features" as well. Like how the owner of the root of a volume gets
extra rights... And the system:ptsviewers group... and the refresh time
on IP-based ACLs... (I could have missed all this in the pages of
documentation though, and yes, I know its being worked on.)
At least the AFS3 file system protocols are documented in source code
and a few postscript files. There is no complete documentation for
CIFS. If there was, perhaps I could finish implementing the pieces
that the AFS SMB Server does not yet support.
Post by Christopher D. Clausen
Well, not everyone here at UIUC can afford RAID setups. I've worked at
several departments where the file-server is just a desktop machine with
a single hard drive. Convincing them to spend money on an entire
redundant server is easier than on expensive disk. ("I got this 300GB
USB drive for $200. Why do you need $7000 for an array?") We both know
that RAID is good, and if you have the money use RAID with a
replicatation partner.
What about software RAID? At the very least you can implement that for
the same $200.
Post by Christopher D. Clausen
RAID doesn't help if a processor or a power supply goes bad, or on OS
upgrade freaks out or whatever. As in, its not protection against
SERVER downtime. It protection against only disk failures. (And being
in a place that uses 3 year old Dell desktops as fileservers, this is a
concern.)
No its not but frequently hardware failures are not day zero events.
Most of the time there are bad disk blocks, memory corruption errors,
blue screens that indicate that something is wrong before the hardware
finally goes.

The benefit of AFS is that you can migrate the volumes while they are
in use to another server so that you can perform the hardware
maintenance without disturbing your users.
Post by Christopher D. Clausen
Only the user volumes are read-write? All of my important data is in
user volumes. Read-only data is easy to keep online, just replicate it.
The important stuff is what people are working on RIGHT NOW and if its
down bad things happen.
The Windows replication model in many cases is only as good as backup
volumes or read-only clones that are taken with frequent snapshots.
This is especially true if the user is using client side caching.
Post by Christopher D. Clausen
Warning: I am a Windows person.
So are Rodney and I.
Post by Christopher D. Clausen
I'm not at an enterprise. I'm at a University. My systems at home are
more advanced than most of the "production" services I find in use
throughout campus. (Not at the campus IT level, but at a department
level: single fileserver, 100 clients max.)
I would really like to have the resources to get the AFS file servers
in a state they can be deployed on Windows Servers or even XP
workstations but I drift...
Post by Christopher D. Clausen
Yes, this was annoying. Had to create local partition "S:" (for
software) and install things there, then copy into AFS and mount the
path as S: on each client machine. Same trick for Dfs installs as well.
Almost everything works using this technique. The problem appears to be
mainly the installers, not the apps themselves. Got to watch things
that install as system services or explorer extensions though. That
causes PAIN. Unless of course you ACL everything system:anyuser rl so
that the system can read it. Probably not legal to do in all
instances... but that's another matter...
Another problem is 100BASE networks are at least 4 times slower than
hard drives. Who wants to wait for that? And, I have 8GBs of software
installed on my Windows machines. I believe the Windows AFS cache size
limit is about 1.3GBs. Say all you want about bloated software, but its
a fact of life.
1.3GB is a limitation of the Windows 32-bit process space. Switch to
64-bit Windows and use a 1TB caches.
Post by Christopher D. Clausen
Do you really run Visual Studio and AutoCAD directly out of AFS?
Um. Yes. and ProEngineer. In fact, Rodney has more than 200
engineering applications running out of AFS. I know because I use
his environment to help test each release of OAFW.
Post by Christopher D. Clausen
I only have about 20 machines that I directly maintain. Again, most of
the data is in user volumes that people access from their own computers.
And the vos release stuff is annoying as well. Why should I need to
write an app to handle allowing users to replicate volumes? What about
the user education issue? "Oh the website isn't up to date b/c no one
vos released it yet."
How is this different from "the website isn't up to date because no one
checked in and released the changes yet"? That's an example from
Microsoft's FrontPage. People have to be trained to use software. Its
just a fact of life.

You have to give ACL permissions to the the end users to update the
web pages inside FrontPage. Why wouldn't you have to grant ACLs to do
so for their web sites?
Post by Christopher D. Clausen
Yes, having separate RO and RW paths is good in some cases. Sometimes
it isn't.
I'm not arguing that having RW replication is not a good thing. Its
just really hard to get it right and provide the performance that most
people would prefer. The fact that Windows allows users of the same
data on two different servers to see different versions of the same
file is a serious problem for many applications. I am really
intrigued to know what Microsoft is doing in this case since all of
their locking mechanisms use a mandatory model and require data consistency.
Post by Christopher D. Clausen
Oh I agree. Your view of things and some other view may be different.
I'm just supplying the reasons why someone may choose MS Dfs over
OpenAFS. (Having already done the research and picked AFS.)
If you have a Windows only environment with limited support staff by
all means use Dfs. If you need to only allow your own users to access
it and are willing to perform all of the identity management then by
all means use Dfs.
Post by Christopher D. Clausen
Being a Windows admin, I concur with that. Why waste the fileserver
serving out the same exact app everyday? Why not utilize it for files
that people actually need to share? (I'm on a 10BASE network right now.
I can assure you that there would be much time wasted waiting for Office
to startup out of a network filesystem.)
and many Windows servers have fallen over because 100 users have all
logged in at the same time and started up the pre-configured set of
applications over the network.
Post by Christopher D. Clausen
Well, we have no money for support of any kind and for us both Microsoft
products and OpenAFS are free so its easy to try and compare.
you just pay in a different way.
Post by Christopher D. Clausen
Most departments here at UIUC can easily afford $2000 for an additional
fileserver for replication but cannot afford SAN gear, even low-cost
iSCSI stuff. In the end, it comes down to what you can do with the
money you have. Dfs is still better than just plain CIFS on a single
fileserver.
Or what people think they can afford.
Post by Christopher D. Clausen
Post by Rodney M Dyer
Like you said, Dfs is primarily an all Windows technology anyway. If
you need anything heterogeneous then you must turn to AFS.
This is entirely true. Again, some places are all Windows where Dfs
would make sense. I believe this is why you see more universities using
AFS, they tend to allow more freedom in IT. Corporations pick a
standard and that's it.
<<CDC
Corporation A chooses 'a' and corporation B chooses 'b' and then A buy B
and now they have both or A needs to work with B and now they must
support both. Not as different as the University as one might think.

Jeffrey Altman
Hendrik Hoeth
2006-03-06 22:53:44 UTC
Permalink
Hi,
Post by Jeffrey Altman
What about software RAID? At the very least you can implement that
for the same $200.
I've seen too many software raids (x86, any linux 2.4; never tried 2.6
anymore) corrupt the files under high load, so I trust them less than I
trust a single USB harddisk. Once we even had a commercial 7k$ raid here
for testing, which was based on linux software raid. After two hours of
copying files NOT EVEN A SINGLE FILE had the right md5sum. The
salesperson who watched us became very quiet ... needless to say we
didn't buy it.

Cheers,

Hendrik
--
Fashion is a form of ugliness so intolerable that we have to alter
it every six months. -- Oscar Wilde
KELEMEN Peter
2006-03-07 21:50:45 UTC
Permalink
Post by Hendrik Hoeth
I've seen too many software raids (x86, any linux 2.4; never
tried 2.6 anymore) corrupt the files under high load, so I trust
them less than I trust a single USB harddisk. [...]
That's very interesting, what kind of load do you expose the RAID
to in order to trigger corruption? I've seen many software RAID
arrays as well, and not a single corruption (due to the RAID layer
itself).

Peter
--
.+'''+. .+'''+. .+'''+. .+'''+. .+''
Kelemen Péter / \ / \ ***@cern.ch
.+' `+...+' `+...+' `+...+' `+...+'
Hendrik Hoeth
2006-03-07 23:37:45 UTC
Permalink
Hi,
Post by Hendrik Hoeth
I've seen too many software raids (x86, any linux 2.4; never
tried 2.6 anymore) corrupt the files under high load, so I trust
them less than I trust a single USB harddisk. [...]
That's very interesting, what kind of load do you expose the RAID to
in order to trigger corruption? I've seen many software RAID arrays
as well, and not a single corruption (due to the RAID layer itself).
A single saturated gigabit link should be enough to trigger it. The
first time we observed problems with corrupt files on software raids was
on a machine with a single gigabit link when a handful clients accessed
files simultaneously. After a bit of experimenting we traced it down to
high-load situations, and then we could easily reproduce the problem by
copying files from several machines onto the raid.

When we got to test the commercial storage I was talking about, we
already had burned our fingers with software raids, and thus we knew
what to try. This system was supposed to act as NFS server for a 500
node cluster and it provided two gigabit interfaces. We connected the
interfaces to two different switches of the cluster and from 50 nodes
per interface we copied large (1G) files to the raid. The load on the
machine was acceptable, the speed was ok, but in the end not a single
file had the right md5sum and in the kernel log we found hundreds of
error messages about the software raid.

I've never tried it on linux 2.6 though.

Cheers,

Hendrik
--
Fashion is a form of ugliness so intolerable that we have to alter
it every six months. -- Oscar Wilde
Jeffrey Hutzelman
2006-03-07 21:00:45 UTC
Permalink
On Monday, March 06, 2006 09:17:23 AM -0600 "Christopher D. Clausen"
Post by Christopher D. Clausen
Post by Rodney M Dyer
It's kind of hard to explain, but Windows people and Unix people think
differently about what they are serving out. AFS grew up in a world
where Unix admins wanted to distribute executable "applications" as
well as data from a single name space tree. In the enterprise, Unix
workstations actually run their applications off of the network.
Warning: I am a Windows person.
I'm not at an enterprise. I'm at a University.
So am I. A University _is_ an enterprise, and a challenging one at that,
because of the highly heterogeneous community. At Carnegie Mellon, we use
an enterprise-grade distributed filesystem (AFS), and we use it in exactly
the way Jeff described. Here's why:

Back in the 1980's (long before I got to CMU), a few people had a grand
vision. They imagined a world in which every student and staff member had
his or her own small computer. All of these machines would be similar
hardware and run the same software, maintained and distributed by a central
support group, which would also manage the machines so that individual
users wouldn't have to know how. They'd have a friendly graphical
interface, and the ability to run more than one application at the same
time. All the machines would be tied together into a central system, so
that you could sit down in front of any machine, log in, and have access to
all of your data -- just like traditional timesharing systems, except that
you'd have the processor all to yourself, instead of sharing it with lots
of other people. In addition, there would be locations around campus with
groups of workstations, which would provide easy access to computing
facilities for those who didn't yet have computers of their own, or were
not near their own machines (think students between classes).

Sound familiar? It should; these are basic characteristics common to
nearly every enterprise (corporate, university, etc) computing environment
today, to one degree or another. The difference is that in the 1980's, the
basic components of such a system simply didn't exist. So those visionary
folks (and, independently, similar groups at several other universities)
pitched their idea, got support and funding from various sources, and set
out to build a distributed computing environment.

In those days, disks were small and expensive and computers were relatively
slow, so there was a limit to how much data could be stored on a single
server. Supporting the data-storage needs of an entire university would
require a system that supported multiple servers relatively easily. Also,
affordable workstations had very little disk -- often not enough even for
the complete operating system, let alone the complex software components
and applications which would be part of the new system. So, it had to be
possible to run applications and even most of the operating system from
disks attached to a server somewhere else. To meet these needs, they built
a distributed filesystem (vice), which is a direct ancestor of OpenAFS.

The original plan called for central fileservers to store user data and
master copies of software, and read-only replica servers in each cluster
(lab), to provide operating system and applications software for machines
in that cluster. You may think that 100Mbps networks are slow, but trust
me, they're blindingly fast compared to 4Mbps token ring. By the time I
got here in 1990 or so, 10Mbps coaxial ethernet was common, and the
server-per-cluster idea had been basically abandoned in favor of multiple
centrally-located replicas.


The point is this - in those days, disk was scarce enough that you _had_ to
run not only applications but the OS itself from a central location. The
folks working on the Andrew project weren't the only ones to reach this
conclusion; several other institutions (MIT and the University of Michigan
both come to mind) developed their own filesystems and used them in exactly
the same way. Today, we still do, as do many other institutions, though
most have given up their own filesystems in favor of AFS (just as we gave
up on the Andrew windowing system in favor of X Windows).


-- Jeffrey T. Hutzelman (N3NHS) <jhutz+@cmu.edu>
Sr. Research Systems Programmer
School of Computer Science - Research Computing Facility
Carnegie Mellon University - Pittsburgh, PA
Christopher D. Clausen
2006-03-07 21:55:30 UTC
Permalink
Post by Jeffrey Hutzelman
On Monday, March 06, 2006 09:17:23 AM -0600 "Christopher D. Clausen"
Post by Christopher D. Clausen
Post by Rodney M Dyer
It's kind of hard to explain, but Windows people and Unix people
think differently about what they are serving out. AFS grew up in
a world where Unix admins wanted to distribute executable
"applications" as well as data from a single name space tree. In
the enterprise, Unix workstations actually run their applications
off of the network.
Warning: I am a Windows person.
I'm not at an enterprise. I'm at a University.
So am I. A University _is_ an enterprise, and a challenging one at
that, because of the highly heterogeneous community. At Carnegie
Mellon, we use an enterprise-grade distributed filesystem (AFS), and
UIUC's shared storage (if you can call it that) is 100MB of WebDAV space
per student: http://www.cites.uiuc.edu/netfiles/ As such, we don't use
an enterprise-grade filesystem :-(

Since this is essentially useless for home directories, almost every
department runs their own fileserver, possibly independent of the
central Active Directory domain, though usually not. Since there is
only a single a Windows or Mac OS fileserver, it makes sense to have
machines and the server joined to campus Active Directory to push the
password services to the central IT organization.

We don't just have a heterogeneous community, we have a heterogeneous IT
structure.

In such a model, it makes sense for pre-existing Windows file servers to
be hooked in to a central directory structure like AD and use a shared
tree for storage like MS Dfs. It is also very easy to purchase and
setup a single additional Windows server as a replica of the first one
for high-availability and a backup at some level.

That is all I am suggesting.
Post by Jeffrey Hutzelman
Back in the 1980's (long before I got to CMU), a few people had a
grand vision. They imagined a world in which every student and staff
member had his or her own small computer. All of these machines
would be similar hardware and run the same software, maintained and
distributed by a central support group, which would also manage the
machines so that individual users wouldn't have to know how.
I think this is where UIUC differs. The problem is that there isn't a
single entity managing all computers (as is the case at most schools.)
There is a centrally maintained Active Directory domain, with seperate
OUs delegated to various other autonomous groups:
http://www.ad.uiuc.edu/

<<CDC
--
Christopher D. Clausen
***@UIUC SysAdmin
Jeffrey Altman
2006-03-07 22:13:39 UTC
Permalink
Post by Christopher D. Clausen
I think this is where UIUC differs. The problem is that there isn't a
single entity managing all computers (as is the case at most schools.)
There is a centrally maintained Active Directory domain, with seperate
OUs delegated to various other autonomous groups: http://www.ad.uiuc.edu/
<<CDC
I don't know of any University that has a single IT group that manages
all computers. You are lucky if you have one group that manages the
network. Not even CMU and MIT have a single group.

You are simply choosing a different approach to solving the same
problems than those schools have done.

Jeffrey Altman
Steve Simmons
2006-03-08 18:21:15 UTC
Permalink
Post by Christopher D. Clausen
Post by Jeffrey Hutzelman
Back in the 1980's (long before I got to CMU), a few people had a
grand vision. They imagined a world in which every student and staff
member had his or her own small computer. All of these machines
would be similar hardware and run the same software, maintained and
distributed by a central support group, which would also manage the
machines so that individual users wouldn't have to know how.
I think this is where UIUC differs. The problem is that there
isn't a single entity managing all computers (as is the case at
most schools.) There is a centrally maintained Active Directory
domain, with seperate OUs delegated to various other autonomous
groups: http://www.ad.uiuc.edu/
It sounds like UIUC is similar to University of Michigan, where the
administration strongly encourages decentralization. At one time UofM
had more than a half-dozen AFS cells in it, each run by different
departments with different rule sets.

For fiddily stupid reasons, the main campus AFS service (called IFS,
as I slip into UMisms below) charged too much for IFS space and
didn't supply much in terms of large quotas. This changed a year or
two back, and the result is that departments are finding they can get
AFS space from the IFS service cheaper than they can run it
themselves. Of the existing campus cells I know of, all but two are
scheduled to or in the process of converting to using central IFS.
One of those is the research cell at CITI, which will probably never
go away, and one is at a school which has some severe restrictions on
what they can do with their data due to medical and research privacy
regulations.

Since the changes to storage charges and quotas, IFS space has been
growing spectacularly. Our usage is up 50% in the last 75 days. The
advantages of a campus-wide filestore (especially when backed by what
is effectively a campus-wide single sign-on; see <http://
www.umich.edu/~umweb/software/cosign/>) are huge.

AFS/IFS has historically not supported Windows nor MacOS well. This
seems to be changing, but other folks can speak to that better than
I. If/when it does, I expect IFS to start making more inroads in
those communities on campus.

But there will always be a need for local file store, where 'local'
is a deliberately vague word. We have departments or groups which
need multiple terabytes of disk for a weekend of heavy-duty data
crunching. No network-attached file server is going to serve that
data across gigabit links and give adequate performance; SANs might
but who can afford to borrow a SAN for a weekend? Only departmental-
level storage is going to meet those requirements.

I guess my point is that AFS is *an* answer to file system issues,
but there is not and likely will never be an answer to *all* file
system issues, regardless of centralized vs decentralize or
commercial vs. academic. Bleeding edge research using large bulk data
needs its store 'close to' the CPUs, as does corporate data mining.
Regulatory issues sometimes mean that file store cannot be
centralized without putting the entire central support staff (and
file store) under onerous requirements. Sometime Micro$oft trumps
everything. There's a mix, there's always going to be a mix.
Steve Roseman
2006-03-13 16:21:06 UTC
Permalink
Forgive the paranoia, but, I have 2 quick questions, requiring Yes or No
answers. (3 servers, Linux 2.4.22, Fedora Core 1)

1) Can I go from 1.2.11 File and DB servers to 1.4.0 by replacing the
/usr/afs binaries and restarting?

2) Can I do one system at a time (one a day)?

Thanks,
Steve

-----------------------------------------------------------------------------
Stephen G. Roseman
Lehigh University
***@lehigh.edu
Derrick J Brashear
2006-03-13 16:35:08 UTC
Permalink
Post by Steve Roseman
Forgive the paranoia, but, I have 2 quick
questions, requiring Yes or No answers. (3
servers, Linux 2.4.22, Fedora Core 1)
1) Can I go from 1.2.11 File and DB servers to
1.4.0 by replacing the /usr/afs binaries and
restarting?
yup (do vol/fs/salvager at once)
Post by Steve Roseman
2) Can I do one system at a time (one a day)?
yup

Derrick
Jeffrey Hutzelman
2006-03-15 16:32:43 UTC
Permalink
On Monday, March 13, 2006 11:35:08 AM -0500 Derrick J Brashear
Post by Derrick J Brashear
Post by Steve Roseman
Forgive the paranoia, but, I have 2 quick
questions, requiring Yes or No answers. (3
servers, Linux 2.4.22, Fedora Core 1)
1) Can I go from 1.2.11 File and DB servers to
1.4.0 by replacing the /usr/afs binaries and
restarting?
yup (do vol/fs/salvager at once)
... but instead, you should think about waiting for 1.4.1, which should be
out fairly soon.

ted creedon
2006-03-07 00:51:43 UTC
Permalink
Many large corporations don't pick standards or try to enforce them.

It's a futile endeavor and the users generally have the last say.

Pick a file system that users want to use, or can be encouraged to use.

One might even have to wear a sales hat for a while....

tedc
Continue reading on narkive:
Loading...