openwrt/staging/blogic.git
15 years agoceph: remove bad calls to ceph_con_shutdown
Sage Weil [Wed, 18 Nov 2009 19:22:36 +0000 (11:22 -0800)]
ceph: remove bad calls to ceph_con_shutdown

We want to ceph_con_close when we're done with the connection, before
the ref count reaches 0.  Once it does, do not call ceph_con_shutdown,
as that takes the con mutex and may sleep, and besides that is
unnecessary.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: fix page invalidation deadlock
Sage Weil [Thu, 12 Nov 2009 23:07:24 +0000 (15:07 -0800)]
ceph: fix page invalidation deadlock

We occasionally want to make a best-effort attempt to invalidate cache
pages without fear of blocking.  If this fails, we fall back to an async
invalidate in another thread.

Use invalidate_mapping_pages instead of invalidate_inode_page2, as that
will skip locked pages, and not deadlock.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: build cleanly without CONFIG_DEBUG_FS
Sage Weil [Thu, 12 Nov 2009 23:05:52 +0000 (15:05 -0800)]
ceph: build cleanly without CONFIG_DEBUG_FS

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: pr_info when mds reconnect completes
Sage Weil [Wed, 11 Nov 2009 23:50:12 +0000 (15:50 -0800)]
ceph: pr_info when mds reconnect completes

This helps the user know what's going on during the (involved) reconnect
process.  They already see when the mds fails and reconnect starts.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: initialize i_size/i_rbytes on snapdir
Sage Weil [Wed, 11 Nov 2009 23:22:37 +0000 (15:22 -0800)]
ceph: initialize i_size/i_rbytes on snapdir

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: exclude snapdir from readdir results
Sage Weil [Wed, 11 Nov 2009 23:21:27 +0000 (15:21 -0800)]
ceph: exclude snapdir from readdir results

It was hidden from sync readdir, but not the cached dcache version.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: remove recon_gen logic
Sage Weil [Wed, 11 Nov 2009 00:02:23 +0000 (16:02 -0800)]
ceph: remove recon_gen logic

We don't get an explicit affirmative confirmation that our caps reconnect,
nor do we necessarily want to pay that cost.  So, take all this code out
for now.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: separate banner and connect during handshake into distinct stages
Sage Weil [Tue, 10 Nov 2009 22:34:36 +0000 (14:34 -0800)]
ceph: separate banner and connect during handshake into distinct stages

We need to make sure we only swab the address during the banner once.  So
break process_banner out of process_connect, and clean up the surrounding
code so that these are distinct phases of the handshake.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: do not confuse stale and dead (unreconnected) caps
Sage Weil [Mon, 9 Nov 2009 20:05:48 +0000 (12:05 -0800)]
ceph: do not confuse stale and dead (unreconnected) caps

We were using the cap_gen to track both stale caps (caps that timed out
due to temporarily losing touch with the mds) and dead caps that did not
reconnect after an MDS failure.  Introduce a recon_gen counter to track
reconnections to restarted MDSs and kill dead caps based on that instead.

Rename gen to cap_gen while we're at it to make it more clear which is
which.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: make CRUSH hash function a bucket property
Sage Weil [Sun, 8 Nov 2009 04:18:22 +0000 (20:18 -0800)]
ceph: make CRUSH hash function a bucket property

Make the integer hash function a property of the bucket it is used on.  This
allows us to gracefully add support for new hash functions without starting
from scatch.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: make object hash a pg_pool property
Sage Weil [Sat, 7 Nov 2009 05:55:25 +0000 (21:55 -0800)]
ceph: make object hash a pg_pool property

The object will be hashed to a placement seed (ps) based on the pg_pool's
hash function.  This allows new hashes to be introduced into an existing
object store, or selection of a hash appropriate to the objects that
will be stored in a particular pool.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: use strong hash function for mapping objects to pgs
Sage Weil [Sat, 7 Nov 2009 00:44:05 +0000 (16:44 -0800)]
ceph: use strong hash function for mapping objects to pgs

We were using the (weak) dcache hash function, but it was leaving lower
bits consecutive for consecutive (inode) objects.  We really want to make
the object to pg mapping random and uniform, so use a proper hash function
here.

This is Robert Jenkin's public domain hash function (with some minor
cleanup):
http://burtleburtle.net/bob/hash/evahash.html

This is a protocol revision.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: make CRUSH hash functions non-inline
Sage Weil [Sat, 7 Nov 2009 00:39:26 +0000 (16:39 -0800)]
ceph: make CRUSH hash functions non-inline

These are way to big to be inline.  I missed crush/* when doing the inline
audit for akpm's review.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: clean up 'osd%d down' console msg
Sage Weil [Fri, 6 Nov 2009 21:57:49 +0000 (13:57 -0800)]
ceph: clean up 'osd%d down' console msg

No ceph prefix.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: convert port endianness
Sage Weil [Wed, 4 Nov 2009 19:46:35 +0000 (11:46 -0800)]
ceph: convert port endianness

The port is informational only, but we should make it correct.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: fix sparse endian warning
Sage Weil [Wed, 4 Nov 2009 19:40:05 +0000 (11:40 -0800)]
ceph: fix sparse endian warning

Use the __le macro, even though for -1 it doesn't matter.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: fix endian conversions for ceph_pg
Sage Weil [Wed, 4 Nov 2009 19:39:12 +0000 (11:39 -0800)]
ceph: fix endian conversions for ceph_pg

The endian conversions don't quite work with the old union ceph_pg.  Just
make it a regular struct, and make each field __le.  This is simpler and it
has the added bonus of actually working.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: use fixed endian encoding for ceph_entity_addr
Sage Weil [Tue, 3 Nov 2009 23:17:56 +0000 (15:17 -0800)]
ceph: use fixed endian encoding for ceph_entity_addr

We exchange struct ceph_entity_addr over the wire and store it on disk.
The sockaddr_storage.ss_family field, however, is host endianness.  So,
fix ss_family endianness to big endian when sending/receiving over the
wire.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: init/destroy bdi in client create/destroy helpers
Sage Weil [Mon, 2 Nov 2009 17:32:47 +0000 (09:32 -0800)]
ceph: init/destroy bdi in client create/destroy helpers

This keeps bdi setup/teardown in line with client life cycle.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agocrush: always return a value from crush_bucket_choose
Sage Weil [Mon, 2 Nov 2009 01:53:24 +0000 (17:53 -0800)]
crush: always return a value from crush_bucket_choose

Even when we encounter a corrupt bucket.  We still BUG().  This fixes
the warning

fs/ceph/crush/mapper.c: In function 'crush_choose':
fs/ceph/crush/mapper.c:352: warning: control may reach end of non-void function
'crush_bucket_choose' being inlined

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: fix uninitialized err variable
Sage Weil [Mon, 2 Nov 2009 01:51:15 +0000 (17:51 -0800)]
ceph: fix uninitialized err variable

Fixes warning
fs/ceph/xattr.c: In function '__build_xattrs':
fs/ceph/xattr.c:353: warning: 'err' may be used uninitialized in this function

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: fix intra strip unit length calculation
Noah Watkins [Fri, 30 Oct 2009 19:57:30 +0000 (12:57 -0700)]
ceph: fix intra strip unit length calculation

Commit 645a102581b3639836b17d147c35d574fd6e8267 fixes calculation of object
offset for layouts with multiple stripes per object. This updates the
calculation of the length written to take into account multiple stripes per
object.

Signed-off-by: Noah Watkins <noah@noahdesu.com>
Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: fix object striping calculation for non-default striping schemes
Sage Weil [Wed, 28 Oct 2009 22:15:05 +0000 (15:15 -0700)]
ceph: fix object striping calculation for non-default striping schemes

We were incorrectly calculationing of object offset.  If we have multiple
stripe units per object, we need to shift to the start of the current
su in addition to the offset within the su.

Also rename bno to ono (object number) to avoid some variable naming
confusion.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: correct comment to match striping calculation
Sage Weil [Wed, 28 Oct 2009 21:57:25 +0000 (14:57 -0700)]
ceph: correct comment to match striping calculation

The object extent offset is the file offset _modulo_ the stripe unit.
The code was correct, the comment was wrong.

Reported-by: Noah Watkins <jayhawk@soe.ucsc.edu>
Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: remove redundant use of le32_to_cpu
Noah Watkins [Wed, 28 Oct 2009 21:04:48 +0000 (14:04 -0700)]
ceph: remove redundant use of le32_to_cpu

Using stripe unit size calculated and saved on the stack to avoid
a redundant call to le32_to_cpu.

Signed-off-by: Noah Watkins <noah@noahdesu.com>
Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: replace list_entry with container_of
Noah Watkins [Wed, 28 Oct 2009 18:54:49 +0000 (11:54 -0700)]
ceph: replace list_entry with container_of

Usage of non-list.h list_entry function for container_of
functionality replaced with direct use of container_of.

Signed-off-by: Noah Watkins <noah@noahdesu.com>
Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: allocate and parse mount args before client instance
Sage Weil [Tue, 27 Oct 2009 18:50:50 +0000 (11:50 -0700)]
ceph: allocate and parse mount args before client instance

This simplifies much of the error handling during mount.  It also means
that we have the mount args before client creation, and we can initialize
based on those options.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: fix, clean up string mount arg parsing
Sage Weil [Tue, 27 Oct 2009 17:19:28 +0000 (10:19 -0700)]
ceph: fix, clean up string mount arg parsing

Clearly demark int and string argument options, and do not try to convert
string arguments to ints.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: silence uninitialized variable warning
Sage Weil [Tue, 27 Oct 2009 05:06:22 +0000 (22:06 -0700)]
ceph: silence uninitialized variable warning

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: reduce parse_mount_args stack usage
Sage Weil [Tue, 27 Oct 2009 05:07:53 +0000 (22:07 -0700)]
ceph: reduce parse_mount_args stack usage

Since we've increased the max mon count, we shouldn't put the addr array
on the parse_mount_args stack.  Put it on the heap instead.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: remove small mon addr limit; use CEPH_MAX_MON where appropriate
Sage Weil [Thu, 22 Oct 2009 17:53:02 +0000 (10:53 -0700)]
ceph: remove small mon addr limit; use CEPH_MAX_MON where appropriate

Get rid of separate max mon limit; use the system limit instead.  This
allows mounts when there are lots of mon addrs provided by mount.ceph (as
with a host with lots of A/AAAA records).

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: move directory size logic to ceph_getattr
Sage Weil [Wed, 21 Oct 2009 18:21:49 +0000 (11:21 -0700)]
ceph: move directory size logic to ceph_getattr

We can't fill i_size with rbytes at the fill_file_size stage without
adding additional checks for directories.  Notably, we want st_blocks
to remain 0 on directories so that 'du' still works.

Fill in i_blocks, i_size specially in ceph_getattr instead.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: v0.17 of client
Sage Weil [Mon, 19 Oct 2009 23:17:31 +0000 (16:17 -0700)]
ceph: v0.17 of client

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: include preferred osd in placement seed
Sage Weil [Mon, 19 Oct 2009 18:41:51 +0000 (11:41 -0700)]
ceph: include preferred osd in placement seed

Mix the preferred osd (if any) into the placement seed that is fed into
the CRUSH object placement calculation.  This prevents all the placement
pgs from peering with the same osds.

Rev the osd client protocol with this change.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: enable readahead
Sage Weil [Fri, 16 Oct 2009 21:44:35 +0000 (14:44 -0700)]
ceph: enable readahead

Initialized bdi->ra_pages to enable readahead.  Use 512KB default.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: move dirty caps code around
Sage Weil [Fri, 16 Oct 2009 01:13:53 +0000 (18:13 -0700)]
ceph: move dirty caps code around

Cleanup only.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: warn on allocation from msgpool with larger front_len
Sage Weil [Thu, 15 Oct 2009 00:36:07 +0000 (17:36 -0700)]
ceph: warn on allocation from msgpool with larger front_len

Pass the front_len we need when pulling a message off a msgpool,
and WARN if it is greater than the pool's size.  Then try to
allocate a new message (to continue without failing).

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: correct subscribe_ack msgpool payload size
Sage Weil [Thu, 15 Oct 2009 00:26:40 +0000 (17:26 -0700)]
ceph: correct subscribe_ack msgpool payload size

Defined a struct for the SUBSCRIBE_ACK, and use that to size
the msgpool.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: flush dirty caps via the cap_dirty list
Sage Weil [Wed, 14 Oct 2009 21:27:38 +0000 (14:27 -0700)]
ceph: flush dirty caps via the cap_dirty list

Previously we were flushing dirty caps by passing an extra flag
when traversing the delayed caps list.  Besides being a bit ugly,
that can also miss caps that are dirty but didn't result in a
cap requeue: notably, mark_caps_dirty().

Separate the flushing into a separate helper, and traverse the
cap_dirty list.

This also brings i_dirty_item in line with i_dirty_caps: we are
on the list IFF caps != 0.  We carry an inode ref IFF
dirty_caps|flushing_caps != 0.

Lose the unused return value from __ceph_mark_caps_dirty().

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: move generic flushing code into helper
Sage Weil [Wed, 14 Oct 2009 21:24:19 +0000 (14:24 -0700)]
ceph: move generic flushing code into helper

Both callers of __mark_caps_flushing() do the same work; move it
into the helper.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: initialize sb->s_bdi, bdi_unregister after kill_anon_super
Sage Weil [Wed, 14 Oct 2009 21:09:07 +0000 (14:09 -0700)]
ceph: initialize sb->s_bdi, bdi_unregister after kill_anon_super

Writeback doesn't work without the bdi set, and writeback on
umount doesn't work if we unregister the bdi too early.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: convert encode/decode macros to inlines
Sage Weil [Wed, 14 Oct 2009 16:59:09 +0000 (09:59 -0700)]
ceph: convert encode/decode macros to inlines

This avoids the fugly pass by reference and makes the code a bit easier
to read.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: add version field to message header
Sage Weil [Tue, 13 Oct 2009 19:55:26 +0000 (12:55 -0700)]
ceph: add version field to message header

This makes it easier for individual message types to indicate
their particular encoding, and make future changes backward
compatible.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: remove unused CEPH_MSG_{OSD,MDS}_GETMAP
Sage Weil [Sat, 10 Oct 2009 04:52:34 +0000 (21:52 -0700)]
ceph: remove unused CEPH_MSG_{OSD,MDS}_GETMAP

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: ignore trailing data in monamp
Sage Weil [Mon, 12 Oct 2009 17:28:13 +0000 (10:28 -0700)]
ceph: ignore trailing data in monamp

This lets us extend the format more easily.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: add file layout validation
Sage Weil [Fri, 9 Oct 2009 23:38:45 +0000 (16:38 -0700)]
ceph: add file layout validation

This tracks updates to code shared with userspace.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: update to mon client protocol v15
Sage Weil [Fri, 9 Oct 2009 23:36:34 +0000 (16:36 -0700)]
ceph: update to mon client protocol v15

The mon request headers now include session_mon information that must
be properly initialized.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: cancel osd requests before resending them
Sage Weil [Fri, 9 Oct 2009 17:31:32 +0000 (10:31 -0700)]
ceph: cancel osd requests before resending them

This ensures we don't submit the same request twice if we are kicking a
specific osd (as with an osd_reset), or when we hit a transient error and
resend.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: reset osd session on fault, not peer_reset
Sage Weil [Fri, 9 Oct 2009 17:29:18 +0000 (10:29 -0700)]
ceph: reset osd session on fault, not peer_reset

The peer_reset just takes longer (until we reconnect and discover the osd
dropped the session... which it will).

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: fail gracefully on corrupt osdmap (bad pg_temp mapping)
Sage Weil [Fri, 9 Oct 2009 05:22:37 +0000 (22:22 -0700)]
ceph: fail gracefully on corrupt osdmap (bad pg_temp mapping)

Return an error and report a corrupt map instead of crying BUG().

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: revoke osd request message on request completion
Sage Weil [Thu, 8 Oct 2009 23:57:16 +0000 (16:57 -0700)]
ceph: revoke osd request message on request completion

If an osd has failed or returned and a request has been sent twice, it's
possible to get a reply and unregister the request while the request
message is queued for delivery.  Since the message references the caller's
page vector, we need to revoke it before completing.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: fix osd request submission race
Sage Weil [Thu, 8 Oct 2009 23:55:47 +0000 (16:55 -0700)]
ceph: fix osd request submission race

The osd request submission path registers the request, drops and retakes
the request_mutex, then sends it to the OSD.  A racing kick_requests could
sent it during that interval, causing the same msg to be sent twice and
BUGing in the msgr.

Fix by only sending the message if it hasn't been touched by other
threads.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: renew mon subscription before it expires
Sage Weil [Thu, 8 Oct 2009 17:25:46 +0000 (10:25 -0700)]
ceph: renew mon subscription before it expires

Be conservative: renew subscription once half the interval has expired.

Do not reuse sub expiration to control hunting.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: fix mdsmap decoding when multiple mds's are present
Sage Weil [Wed, 7 Oct 2009 23:38:19 +0000 (16:38 -0700)]
ceph: fix mdsmap decoding when multiple mds's are present

A misplaced sizeof() around namelen was throwing things off.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: gracefully avoid empty crush buckets
Sage Weil [Wed, 7 Oct 2009 17:59:34 +0000 (10:59 -0700)]
ceph: gracefully avoid empty crush buckets

This avoids a divide by zero when the input and/or map are
malformed.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: include preferred_osd in file layout virtual xattr
Sage Weil [Wed, 7 Oct 2009 17:59:30 +0000 (10:59 -0700)]
ceph: include preferred_osd in file layout virtual xattr

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: show meaningful version on module load
Sage Weil [Wed, 7 Oct 2009 17:59:10 +0000 (10:59 -0700)]
ceph: show meaningful version on module load

Kill the old git revision; print the ceph version and protocol
versions instead.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: document shared files in README
Sage Weil [Tue, 6 Oct 2009 19:20:56 +0000 (12:20 -0700)]
ceph: document shared files in README

Document files shared between kernel and user code trees.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: Kconfig, Makefile
Sage Weil [Tue, 6 Oct 2009 18:31:15 +0000 (11:31 -0700)]
ceph: Kconfig, Makefile

Kconfig options and Makefile.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: debugfs
Sage Weil [Tue, 6 Oct 2009 18:31:14 +0000 (11:31 -0700)]
ceph: debugfs

Basic state information is available via /sys/kernel/debug/ceph,
including instances of the client, fsids, current monitor, mds and osd
maps, outstanding server requests, and hooks to adjust debug levels.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: ioctls
Sage Weil [Tue, 6 Oct 2009 18:31:14 +0000 (11:31 -0700)]
ceph: ioctls

A few Ceph ioctls for getting and setting file layout (striping)
parameters, and learning the identity and network address of the OSD a
given region of a file is stored on.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: nfs re-export support
Sage Weil [Tue, 6 Oct 2009 18:31:13 +0000 (11:31 -0700)]
ceph: nfs re-export support

Basic NFS re-export support is included.  This mostly works.  However,
Ceph's MDS design precludes the ability to generate a (small)
filehandle that will be valid forever, so this is of limited utility.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: message pools
Sage Weil [Tue, 6 Oct 2009 18:31:13 +0000 (11:31 -0700)]
ceph: message pools

The msgpool is a basic mempool_t-like structure to preallocate
messages we expect to receive over the wire.  This ensures we have the
necessary memory preallocated to process replies to requests, or to
process unsolicited messages from various servers.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: messenger library
Sage Weil [Tue, 6 Oct 2009 18:31:13 +0000 (11:31 -0700)]
ceph: messenger library

A generic message passing library is used to communicate with all
other components in the Ceph file system.  The messenger library
provides ordered, reliable delivery of messages between two nodes in
the system.

This implementation is based on TCP.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: snapshot management
Sage Weil [Tue, 6 Oct 2009 18:31:12 +0000 (11:31 -0700)]
ceph: snapshot management

Ceph snapshots rely on client cooperation in determining which
operations apply to which snapshots, and appropriately flushing
snapshotted data and metadata back to the OSD and MDS clusters.
Because snapshots apply to subtrees of the file hierarchy and can be
created at any time, there is a fair bit of bookkeeping required to
make this work.

Portions of the hierarchy that belong to the same set of snapshots
are described by a single 'snap realm.'  A 'snap context' describes
the set of snapshots that exist for a given file or directory.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: capability management
Sage Weil [Tue, 6 Oct 2009 18:31:12 +0000 (11:31 -0700)]
ceph: capability management

The Ceph metadata servers control client access to inode metadata and
file data by issuing capabilities, granting clients permission to read
and/or write both inode field and file data to OSDs (storage nodes).
Each capability consists of a set of bits indicating which operations
are allowed.

If the client holds a *_SHARED cap, the client has a coherent value
that can be safely read from the cached inode.

In the case of a *_EXCL (exclusive) or FILE_WR capabilities, the client
is allowed to change inode attributes (e.g., file size, mtime), note
its dirty state in the ceph_cap, and asynchronously flush that
metadata change to the MDS.

In the event of a conflicting operation (perhaps by another client),
the MDS will revoke the conflicting client capabilities.

In order for a client to cache an inode, it must hold a capability
with at least one MDS server.  When inodes are released, release
notifications are batched and periodically sent en masse to the MDS
cluster to release server state.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: monitor client
Sage Weil [Tue, 6 Oct 2009 18:31:11 +0000 (11:31 -0700)]
ceph: monitor client

The monitor cluster is responsible for managing cluster membership
and state.  The monitor client handles what minimal interaction
the Ceph client has with it: checking for updated versions of the
MDS and OSD maps, getting statfs() information, and unmounting.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: CRUSH mapping algorithm
Sage Weil [Tue, 6 Oct 2009 18:31:11 +0000 (11:31 -0700)]
ceph: CRUSH mapping algorithm

CRUSH is a pseudorandom data distribution function designed to map
inputs onto a dynamic hierarchy of devices, while minimizing the
extent to which inputs are remapped when the devices are added or
removed.  It includes some features that are specifically useful for
storage, most notably the ability to map each input onto a set of N
devices that are separated across administrator-defined failure
domains.  CRUSH is used to distribute data across the cluster of Ceph
storage nodes.

More information about CRUSH can be found in this paper:

    http://www.ssrc.ucsc.edu/Papers/weil-sc06.pdf

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: OSD client
Sage Weil [Tue, 6 Oct 2009 18:31:10 +0000 (11:31 -0700)]
ceph: OSD client

The OSD client is responsible for reading and writing data from/to the
object storage pool.  This includes determining where objects are
stored in the cluster, and ensuring that requests are retried or
redirected in the event of a node failure or data migration.

If an OSD does not respond before a timeout expires, keepalive
messages are sent across the lossless, ordered communications channel
to ensure that any break in the TCP is discovered.  If the session
does reset, a reconnection is attempted and affected requests are
resent (by the message transport layer).

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: MDS client
Sage Weil [Tue, 6 Oct 2009 18:31:09 +0000 (11:31 -0700)]
ceph: MDS client

The MDS (metadata server) client is responsible for submitting
requests to the MDS cluster and parsing the response.  We decide which
MDS to submit each request to based on cached information about the
current partition of the directory hierarchy across the cluster.  A
stateful session is opened with each MDS before we submit requests to
it, and a mutex is used to control the ordering of messages within
each session.

An MDS request may generate two responses.  The first indicates the
operation was a success and returns any result.  A second reply is
sent when the operation commits to disk.  Note that locking on the MDS
ensures that the results of updates are visible only to the updating
client before the operation commits.  Requests are linked to the
containing directory so that an fsync will wait for them to commit.

If an MDS fails and/or recovers, we resubmit requests as needed.  We
also reconnect existing capabilities to a recovering MDS to
reestablish that shared session state.  Old dentry leases are
invalidated.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: address space operations
Sage Weil [Tue, 6 Oct 2009 18:31:09 +0000 (11:31 -0700)]
ceph: address space operations

The ceph address space methods are concerned primarily with managing
the dirty page accounting in the inode, which (among other things)
must keep track of which snapshot context each page was dirtied in,
and ensure that dirty data is written out to the OSDs in snapshort
order.

A writepage() on a page that is not currently writeable due to
snapshot writeback ordering constraints is ignored (it was presumably
called from kswapd).

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: file operations
Sage Weil [Tue, 6 Oct 2009 18:31:08 +0000 (11:31 -0700)]
ceph: file operations

File open and close operations, and read and write methods that ensure
we have obtained the proper capabilities from the MDS cluster before
performing IO on a file.  We take references on held capabilities for
the duration of the read/write to avoid prematurely releasing them
back to the MDS.

We implement two main paths for read and write: one that is buffered
(and uses generic_aio_{read,write}), and one that is fully synchronous
and blocking (operating either on a __user pointer or, if O_DIRECT,
directly on user pages).

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: directory operations
Sage Weil [Tue, 6 Oct 2009 18:31:08 +0000 (11:31 -0700)]
ceph: directory operations

Directory operations, including lookup, are defined here.  We take
advantage of lookup intents when possible.  For the most part, we just
need to build the proper requests for the metadata server(s) and
pass things off to the mds_client.

The results of most operations are normally incorporated into the
client's cache when the reply is parsed by ceph_fill_trace().
However, if the MDS replies without a trace (e.g., when retrying an
update after an MDS failure recovery), some operation-specific cleanup
may be needed.

We can validate cached dentries in two ways.  A per-dentry lease may
be issued by the MDS, or a per-directory cap may be issued that acts
as a lease on the entire directory.  In the latter case, a 'gen' value
is used to determine which dentries belong to the currently leased
directory contents.

We normally prepopulate the dcache and icache with readdir results.
This makes subsequent lookups and getattrs avoid any server
interaction.  It also lets us satisfy readdir operation by peeking at
the dcache IFF we hold the per-directory cap/lease, previously
performed a readdir, and haven't dropped any of the resulting
dentries.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: inode operations
Sage Weil [Tue, 6 Oct 2009 18:31:08 +0000 (11:31 -0700)]
ceph: inode operations

Inode cache and inode operations.  We also include routines to
incorporate metadata structures returned by the MDS into the client
cache, and some helpers to deal with file capabilities and metadata
leases.  The bulk of that work is done by fill_inode() and
fill_trace().

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: super.c
Sage Weil [Tue, 6 Oct 2009 18:31:07 +0000 (11:31 -0700)]
ceph: super.c

Mount option parsing, client setup and teardown, and a few odds and
ends (e.g., statfs).

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: ref counted buffer
Sage Weil [Tue, 6 Oct 2009 18:31:07 +0000 (11:31 -0700)]
ceph: ref counted buffer

struct ceph_buffer is a simple ref-counted buffer.  We transparently
choose between kmalloc for small buffers and vmalloc for large ones.

This is currently used only for allocating memory for xattr data.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: client types
Sage Weil [Tue, 6 Oct 2009 18:31:07 +0000 (11:31 -0700)]
ceph: client types

We first define constants, types, and prototypes for the kernel client
proper.

A few subsystems are defined separately later: the MDS, OSD, and
monitor clients, and the messaging layer.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: on-wire types
Sage Weil [Tue, 6 Oct 2009 18:31:06 +0000 (11:31 -0700)]
ceph: on-wire types

These headers describe the types used to exchange messages between the
Ceph client and various servers.  All types are little-endian and
packed.  These headers are shared between the kernel and userspace, so
all types are in terms of e.g. __u32.

Additionally, we define a few magic values to identify the current
version of the protocol(s) in use, so that discrepancies to be
detected on mount.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoceph: documentation
Sage Weil [Tue, 6 Oct 2009 18:31:05 +0000 (11:31 -0700)]
ceph: documentation

Mount options, syntax.

Signed-off-by: Sage Weil <sage@newdream.net>
15 years agoLinux 2.6.32-rc1
Linus Torvalds [Sun, 27 Sep 2009 21:57:48 +0000 (14:57 -0700)]
Linux 2.6.32-rc1

15 years agoalpha: Fix duplicate <asm/thread_info.h> include
Linus Torvalds [Sun, 27 Sep 2009 21:46:05 +0000 (14:46 -0700)]
alpha: Fix duplicate <asm/thread_info.h> include

.. duplicated by merging the same fix twice, for details see commit
0d9df2515dbceb67d343c0f10fd3ff218380d524 ("Merge
git://git.kernel.org/pub/scm/linux/kernel/git/sam/kbuild-fixes")

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
15 years agotty: Fix regressions caused by commit b50989dc
Dave Young [Sun, 27 Sep 2009 16:00:42 +0000 (16:00 +0000)]
tty: Fix regressions caused by commit b50989dc

The following commit made console open fails while booting:

commit b50989dc444599c8b21edc23536fc305f4e9b7d5
Author: Alan Cox <alan@linux.intel.com>
Date:   Sat Sep 19 13:13:22 2009 -0700

tty: make the kref destructor occur asynchronously

Due to tty release routines run in a workqueue now, error like the
following will be reported while booting:

INIT open /dev/console Input/output error

It also causes hibernation regression to appear as reported at
http://bugzilla.kernel.org/show_bug.cgi?id=14229

The reason is that now there's latency issue with closing, but when
we open a "closing not finished" tty, -EIO will be returned.

Fix it as per the following Alan's suggestion:

  Fun but it's actually not a bug and the fix is wrong in itself as
  the port may be closing but not yet being destructed, in which case
  it seems to do the wrong thing.  Opening a tty that is closing (and
  could be closing for long periods) is supposed to return -EIO.

  I suspect a better way to deal with this and keep the old console
  timing is to split tty->shutdown into two functions.

  tty->shutdown() - called synchronously just before we dump the tty
  onto the waitqueue for destruction

  tty->cleanup() - called when the destructor runs.

  We would then do the shutdown part which can occur in IRQ context
  fine, before queueing the rest of the release (from tty->magic = 0
  ...  the end) to occur asynchronously

  The USB update in -next would then need a call like

       if (tty->cleanup)
               tty->cleanup(tty);

  at the top of the async function and the USB shutdown to be split
  between shutdown and cleanup as the USB resource cleanup and final
  tidy cannot occur synchronously as it needs to sleep.

  In other words the logic becomes

       final kref put
               make object unfindable

       async
               clean it up

Signed-off-by: Dave Young <hidave.darkstar@gmail.com>
[ rjw: Rebased on top of 2.6.31-git, reworked the changelog. ]
Signed-off-by: "Rafael J. Wysocki" <rjw@sisk.pl>
[ Changed serial naming to match new rules, dropped tty_shutdown as per
  comments from Alan Stern  - Linus ]
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
15 years agoACPI: kill "unused variable ‘i’" warning
Linus Torvalds [Sun, 27 Sep 2009 18:58:36 +0000 (11:58 -0700)]
ACPI: kill "unused variable ‘i’" warning

Commit 3d5b6fb47a8e68fa311ca2c3447e7f8a7c3a9cf3 ("ACPI: Kill overly
verbose "power state" log messages") removed the actual use of this
variable, but didn't remove the variable itself, resulting in build
warnings like

  drivers/acpi/processor_idle.c: In function ‘acpi_processor_power_init’:
  drivers/acpi/processor_idle.c:1169: warning: unused variable ‘i’

Just get rid of the now unused variable.

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
15 years agoconst: mark struct vm_struct_operations
Alexey Dobriyan [Sun, 27 Sep 2009 18:29:37 +0000 (22:29 +0400)]
const: mark struct vm_struct_operations

* mark struct vm_area_struct::vm_ops as const
* mark vm_ops in AGP code

But leave TTM code alone, something is fishy there with global vm_ops
being used.

Signed-off-by: Alexey Dobriyan <adobriyan@gmail.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
15 years agoMerge branch 'timers-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git...
Linus Torvalds [Sun, 27 Sep 2009 17:39:04 +0000 (10:39 -0700)]
Merge branch 'timers-for-linus' of git://git./linux/kernel/git/tip/linux-2.6-tip

* 'timers-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip:
  hrtimer: Eliminate needless reprogramming of clock events device

15 years agoMerge branch 'release' of git://git.kernel.org/pub/scm/linux/kernel/git/lenb/linux...
Linus Torvalds [Sun, 27 Sep 2009 17:38:48 +0000 (10:38 -0700)]
Merge branch 'release' of git://git./linux/kernel/git/lenb/linux-acpi-2.6

* 'release' of git://git.kernel.org/pub/scm/linux/kernel/git/lenb/linux-acpi-2.6:
  ACPI: IA64=y ACPI=n build fix
  ACPI: Kill overly verbose "power state" log messages
  ACPI: fix Compaq Evo N800c (Pentium 4m) boot hang regression
  ACPI: Clarify resource conflict message
  thinkpad-acpi: fix CONFIG_THINKPAD_ACPI_HOTKEY_POLL build problem

15 years agoMerge branch 'x86-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git...
Linus Torvalds [Sun, 27 Sep 2009 17:38:34 +0000 (10:38 -0700)]
Merge branch 'x86-fixes-for-linus' of git://git./linux/kernel/git/tip/linux-2.6-tip

* 'x86-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip:
  x86: Fix hwpoison code related build failure on 32-bit NUMAQ

15 years agoACPI: IA64=y ACPI=n build fix
Len Brown [Sun, 27 Sep 2009 08:17:21 +0000 (04:17 -0400)]
ACPI: IA64=y ACPI=n build fix

ia64's sim_defconfig uses CONFIG_ACPI=n
which now #define's acpi_disabled in <linux/acpi.h>

So we shouldn't re-define it here in <asm/acpi.h>

Signed-off-by: Len Brown <len.brown@intel.com>
15 years agoACPI: Kill overly verbose "power state" log messages
Roland Dreier [Thu, 24 Sep 2009 21:52:36 +0000 (14:52 -0700)]
ACPI: Kill overly verbose "power state" log messages

I was recently lucky enough to get a 64-CPU system, so my kernel log
ends up with 64 lines like:

    ACPI: CPU0 (power states: C1[C1] C2[C3])

This is pretty useless clutter because this info is already available
after boot from both /sys/devices/system/cpu/cpu*/cpuidle/state?/ as
well as /proc/acpi/processor/CPU*/power.

So just delete the code that prints the C-states in processor_idle.c.

Signed-off-by: Roland Dreier <rolandd@cisco.com>
Signed-off-by: Len Brown <len.brown@intel.com>
15 years agox86: Fix hwpoison code related build failure on 32-bit NUMAQ
Linus Torvalds [Sat, 26 Sep 2009 16:35:07 +0000 (09:35 -0700)]
x86: Fix hwpoison code related build failure on 32-bit NUMAQ

This build failure triggers:

 In file included from include/linux/suspend.h:8,
                 from arch/x86/kernel/asm-offsets_32.c:11,
                 from arch/x86/kernel/asm-offsets.c:2:
 include/linux/mm.h:503:2: error: #error SECTIONS_WIDTH+NODES_WIDTH+ZONES_WIDTH > BITS_PER_LONG - NR_PAGEFLAGS

Because due to the hwpoison page flag we ran out of page
flags on 32-bit.

Dont turn on hwpoison on 32-bit NUMA (it's rare in any
case).

Also clean up the Kconfig dependencies in the generic MM
code by introducing ARCH_SUPPORTS_MEMORY_FAILURE.

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
15 years agoACPI: fix Compaq Evo N800c (Pentium 4m) boot hang regression
Zhao Yakui [Sun, 27 Sep 2009 07:30:51 +0000 (03:30 -0400)]
ACPI: fix Compaq Evo N800c (Pentium 4m) boot hang regression

Don't disable ARB_DISABLE when the familary ID is 0x0F.

http://bugzilla.kernel.org/show_bug.cgi?id=14211

This was a 2.6.31 regression, and so this patch
needs to be applied to 2.6.31.stable

Signed-off-by: Zhao Yakui <yakui.zhao@intel.com>
Signed-off-by: Len Brown <len.brown@intel.com>
15 years agoACPI: Clarify resource conflict message
Jean Delvare [Tue, 8 Sep 2009 13:31:46 +0000 (15:31 +0200)]
ACPI: Clarify resource conflict message

The message "ACPI: Device needs an ACPI driver" is misleading. The
device _may_ need an ACPI driver, if the BIOS implemented a custom
API for the device in question (which, AFAIK, can't be checked.) If
not, then either a generic ACPI driver may be used (for example
"thermal"), or nothing can be done (other than a white list).

I propose to reword the message to:

ACPI: If an ACPI driver is available for this device, you should use
it instead of the native driver

which I think is more correct. Comments and suggestions welcome.

I also added a message warning about possible problems and system
instability when users pass acpi_enforce_resources=lax, as suggested
by Len.

Signed-off-by: Jean Delvare <jdelvare@suse.de>
Cc: Thomas Renninger <trenn@suse.de>
Cc: Alan Jenkins <sourcejedi.lkml@googlemail.com>
Signed-off-by: Len Brown <len.brown@intel.com>
15 years agothinkpad-acpi: fix CONFIG_THINKPAD_ACPI_HOTKEY_POLL build problem
Henrique de Moraes Holschuh [Sun, 27 Sep 2009 00:42:49 +0000 (21:42 -0300)]
thinkpad-acpi: fix CONFIG_THINKPAD_ACPI_HOTKEY_POLL build problem

Fix this problem when CONFIG_THINKPAD_ACPI_HOTKEY_POLL is undefined:

  CHECK   drivers/platform/x86/thinkpad_acpi.c
drivers/platform/x86/thinkpad_acpi.c:1968:21: error: not an lvalue
  CC [M]  drivers/platform/x86/thinkpad_acpi.o
drivers/platform/x86/thinkpad_acpi.c: In function 'tpacpi_hotkey_driver_mask_set':
drivers/platform/x86/thinkpad_acpi.c:1968: error: lvalue required as left operand of assignment

Reported-by: Noah Dain <noahdain@gmail.com>
Reported-by: Audrius Kazukauskas <audrius@neutrino.lt>
Signed-off-by: Henrique de Moraes Holschuh <hmh@hmh.eng.br>
Signed-off-by: Len Brown <len.brown@intel.com>
15 years agoMerge branch 'release' of git://git.kernel.org/pub/scm/linux/kernel/git/aegl/linux-2.6
Linus Torvalds [Sat, 26 Sep 2009 20:39:21 +0000 (13:39 -0700)]
Merge branch 'release' of git://git./linux/kernel/git/aegl/linux-2.6

* 'release' of git://git.kernel.org/pub/scm/linux/kernel/git/aegl/linux-2.6:
  [IA64] implement ticket locks for Itanium

15 years agoMerge git://git.kernel.org/pub/scm/linux/kernel/git/sam/kbuild-fixes
Linus Torvalds [Sat, 26 Sep 2009 17:51:54 +0000 (10:51 -0700)]
Merge git://git./linux/kernel/git/sam/kbuild-fixes

* git://git.kernel.org/pub/scm/linux/kernel/git/sam/kbuild-fixes:
  alpha: fix build after vmlinux.lds.S cleanup
  mips: fix build of vmlinux.lds

15 years agoMerge branch 'for-linus' of git://git.o-hand.com/linux-rpurdie-leds
Linus Torvalds [Sat, 26 Sep 2009 17:50:47 +0000 (10:50 -0700)]
Merge branch 'for-linus' of git://git.o-hand.com/linux-rpurdie-leds

* 'for-linus' of git://git.o-hand.com/linux-rpurdie-leds:
  leds: move leds-clevo-mail's probe function to .devinit.text
  leds: Fix indentation in LEDS_LP3944 Kconfig entry
  leds: Fix LED names
  leds: Fix leds-pca9532 whitespace issues
  leds: fix coding style in worker thread code for ledtrig-gpio.
  leds: gpio-leds: fix typographics fault
  leds: Add WM831x status LED driver

15 years agoMerge branch 'for-linus' of git://git.o-hand.com/linux-rpurdie-backlight
Linus Torvalds [Sat, 26 Sep 2009 17:49:42 +0000 (10:49 -0700)]
Merge branch 'for-linus' of git://git.o-hand.com/linux-rpurdie-backlight

* 'for-linus' of git://git.o-hand.com/linux-rpurdie-backlight:
  backlight: new driver for ADP5520/ADP5501 MFD PMICs
  backlight: extend event support to also support poll()
  backlight/eeepc-laptop: Update the backlight state when we change brightness
  backlight/acpi: Update the backlight state when we change brightness
  backlight: Allow drivers to update the core, and generate events on changes
  backlight: switch to da903x driver to dev_pm_ops
  backlight: Add support for the Avionic Design Xanthos backlight device.
  backlight: spi driver for LMS283GF05 LCD
  backlight: move hp680-bl's probe function to .devinit.text
  backlight: Add support for new Apple machines.
  backlight: mbp_nvidia_bl: add support for MacBookAir 1,1
  backlight: Add WM831x backlight driver

Trivial conflicts due to '#ifdef CONFIG_PM' differences in
drivers/video/backlight/da903x_bl.c

15 years agoheaders: kref.h redux
Alexey Dobriyan [Sat, 26 Sep 2009 15:37:22 +0000 (19:37 +0400)]
headers: kref.h redux

* remove asm/atomic.h inclusion from kref.h -- not needed, linux/types.h
  is enough for atomic_t
* remove linux/kref.h inclusion from files which do not need it.

Signed-off-by: Alexey Dobriyan <adobriyan@gmail.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
15 years agoMerge branch 'core-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel...
Linus Torvalds [Sat, 26 Sep 2009 17:15:53 +0000 (10:15 -0700)]
Merge branch 'core-fixes-for-linus' of git://git./linux/kernel/git/tip/linux-2.6-tip

* 'core-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip:
  futex: Add memory barrier commentary to futex_wait_queue_me()
  futex: Fix wakeup race by setting TASK_INTERRUPTIBLE before queue_me()
  futex: Correct futex_q woken state commentary
  futex: Make function kernel-doc commentary consistent
  futex: Correct queue_me and unqueue_me commentary
  futex: Correct futex_wait_requeue_pi() commentary

15 years agoMerge branch 'perf-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel...
Linus Torvalds [Sat, 26 Sep 2009 17:15:33 +0000 (10:15 -0700)]
Merge branch 'perf-fixes-for-linus' of git://git./linux/kernel/git/tip/linux-2.6-tip

* 'perf-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip:
  perf tools: Dont use openat()
  perf tools: Fix buffer allocation
  perf tools: .gitignore += perf*.html
  perf tools: Handle relative paths while loading module symbols
  perf tools: Fix module symbol loading bug
  perf_event, x86: Fix 'perf sched record' crashing the machine
  perf_event: Update PERF_EVENT_FORK header definition
  perf stat: Fix zero total printouts