If you create a two drive raid1 array with one device writemostly, then
fail the readwrite drive, when you add a new device, it will get the
writemostly bit copied out of the remaining device's superblock into
it's own. You can then remove the new drive and readd it as readwrite,
which will work for the readd, but it leaves the stale WriteMostly1 bit
in devflags resulting in the device going back to writemostly on the
next assembly.
The fix is to make sure that A) when we readd a device and we might have
filled the st->sb info from a running device instead of the device being
readded, then clear/set the WriteMostly1 bit in the super1 struct in
addition to setting the disk state (ditto for super0, but slightly
different mechanism) and B) when adding a clean device to an array (when
we most certainly did copy the superblock info from an existing device),
then clear any writemostly bits.
Signed-off-by: Doug Ledford <dledford@redhat.com>
Signed-off-by: NeilBrown <neilb@suse.de>
0.90 arrays can only use up to 4TB per device. So when a larger
device is added, complain a bit. Still allow it if --force is given
as there could be a valid use.
Signed-off-by: NeilBrown <neilb@suse.de>
The loop over all member devices in enough_fd could easily stop
before it had found all devices. This would cause --re-add to
fail incorrectly.
So change the loop to be based on the reported number of devices
in the device - with a safe-guard limit of 1024.
Change some other loops to be more careful too.
Reported-by: "Schmidt, Annemarie" <Annemarie.Schmidt@stratus.com>
Signed-off-by: NeilBrown <neilb@suse.de>
If using an old kernel we should still check if a re-add might be
intended, so we can refuse and require a '--zero' first if it is not
possible.
Signed-off-by: NeilBrown <neilb@suse.de>
Stopping an md array requires that there is no other user of it.
However with udev and udisks and such there can be transient other
users of md devices which can interfere with stopping the array.
If there is a transient users, we really want "mdadm --stop" to wait a
little while and retry.
However if the array is genuinely in-use (e.g. mounted), then we
don't want to wait at all - we want to fail immediately.
So before trying to stop, re-open device with O_EXCL. If this fails
then the device is probably in use, so give up.
If it succeeds, but a subsequent STOP_ARRAY fails, then it is possibly
a transient failure, so try again for a few seconds.
Signed-off-by: NeilBrown <neilb@suse.de>
sync_metadata() requires st->sb to be loaded, otherwise exception is
generated. This fails expansion, because spares cannot be added.
metadata update uses tst instead st pointer, it is better than
loading anchor for st as I proposed previously.
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
Issue observed:
Sporadicaly stopping arrays using "mdadm -Ss" command does not succeded.
Cause:
Writting "inactive" to the array state not succeded- array is busy
(accessed by udev, blkid etc.)
Resolution:
If writing 'inactive' fails, wait and retry again (because it is possibly
a transient failure)
Signed-off-by: Krzysztof Wojcik <krzysztof.wojcik@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
When for ping_monitor() input devnum2devname() is used,
received string pointer should be passed to free() for memory release.
It is not made in several places. This use case should have function
to avoid memory leak.
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
When I separated the 'native metadata' case more cleanly from the
"external metadata" case for adding a drive, I left some 'external'
code in the 'native' case, and didn't copy it to the 'external' case.
When - in the external case - we add to super, we much check for
mdmon first, so we know whether to do the metadata update ourselves
or not, then afterwards call either flush_metadata_updates (to send
to mdmon) or sync_metadata (to do it directly).
Reported-by: Adam Kwolek <adam.kwolek@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
Stopping an md array requires that there is no other user of it.
However with udev and udisks and such there can be transient other
users of md devices which can interfere with stopping the array.
If there is a transient users, we really want "mdadm --stop" to wait a
little while and retry.
However if the array is genuinely in-use (e.g. mounted), then we
don't want to wait at all - we want to fail immediately.
So before trying to stop, re-open device with O_EXCL. If this fails
then the device is probably in use, so give up.
If it succeeds, but a subsequent STOP_ARRAY fails, then it is possibly
a transient failure, so try again for a few seconds.
Signed-off-by: NeilBrown <neilb@suse.de>
add_to_super could use information from the current superblock (ddf
does), so add_to_super for external metadata should be called with
the O_EXCL lock held on the container to ensure the update is complete
before any other process tries to make any changes (like adding
another device to array).
Signed-off-by: NeilBrown <neilb@suse.de>
If an --add is requested and a re-add looks promising but fails or
cannot possibly succeed, then don't try the add. This avoids
inadvertently turning devices into spares when an array is failed but
the devices seem to actually work.
Signed-off-by: NeilBrown <neilb@suse.de>
Loading container may fail if e.g. one of the disks in container
has been detached but udev has not realized the change.
Addition to such array will fail because reading superblock
from one of disks in array fails.
Current message is a bit confusing.
Signed-off-by: Anna Czarnowska <anna.czarnowska@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
If a request to remove all 'failed' or 'detached' devices chooses to
remove the first device, it will not actually try the removal and will
skip any following devices.
This fixes it.
Reported-by: Rémi Rérolle <rrerolle@lacie.com>
Tested-by: Rémi Rérolle <rrerolle@lacie.com>
Signed-off-by: NeilBrown <neilb@suse.de>
If a request to remove all 'failed' or 'detached' devices chooses to
remove the first device, it will not actually try the removal and will
skip any following devices.
This fixes it.
Reported-by: Rémi Rérolle <rrerolle@lacie.com>
Tested-by: Rémi Rérolle <rrerolle@lacie.com>
Signed-off-by: NeilBrown <neilb@suse.de>
Now that write_init_super doesn't close fds any more, we need
to call free_super before the ADD_NEW_DISK ioctl.
Also call free_super before some error returns, for cleanliness.
Signed-off-by: NeilBrown <neilb@suse.de>
We previously closed all 'fds' associated with an array in
write_init_super .. sometimes, and sometimes at bad times.
This isn't neat and free_super is a better place to close them.
So make sure free_super always closes the fds that the metadata
manager kept hold of, and stop closing them in write_init_super.
Also add a few more calls to free_super to make sure they really do
get closed.
Reported-by: Adam Kwolek <adam.kwolek@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
When user wants to add spares to container with raid0 arrays only
it is not possible to update metadata due to lack of running mdmon.
To allow for this direct metadata update by mdadm is used in such case.
Signed-off-by: Krzysztof Wojcik <krzysztof.wojcik@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
This is useful with 1.1 and 1.2 metadata to update the metadata if
the device size has changed.
The same functionality can be achieved by writing to the device size
in sysfs after re-adding normally, but in some cases this might be
easier.
Signed-off-by: NeilBrown <neilb@suse.de>
mdadm --readwrite <subarray> will clear the external readonly flag ('-'
to '/'), but only for redudant arrays. Allow raid0 arrays as well so
the user has a simple helper to control this flag.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
Remove the _t pointer typedef and remove the _s suffix for the
structure,
These things do not help readability.
Signed-off-by: NeilBrown <neilb@suse.de>
Rather than hiding this in the 'st', return it explicitly.
In the one case we still need it, copy it into st where needed.
This will disappear in a future patch.
Signed-off-by: NeilBrown <neilb@suse.de>
Rather than hiding this arg in the 'st' structure, pass it explicitly.
This is a first step to getting rid of 'subarray' from 'supertype'.
The strcpy in open_subarray should have better error checking, but it
will disappear soon so there is little point.
Signed-off-by: NeilBrown <neilb@suse.de.
To accurately detect when an array has been split and is now being
recombined, we need to track which other devices each thinks is
working.
We should never include a device in an array if it thinks that the
primary device has failed.
This patch just allows get_info_super to return a list of devices
and whether they are thought to be working or not.
Signed-off-by: NeilBrown <neilb@suse.de>
If an --add is requested and a re-add looks promising but fails or
cannot possibly succeed, then don't try the add. This avoids
inadvertently turning devices into spares when an array is failed but
the devices seem to actually work.
Signed-off-by: NeilBrown <neilb@suse.de>
If udev is not in use, we create device in /dev when assembling
arrays and remove them when stopping the array.
However it may not always be correct to remove the device. If
the array was started with kernel auto-detect, them mdadm didn't
create anything and so shouldn't remove anything.
We don't record whether we created things, so just don't remove
anything with a 'standard' name. Only remove symlinks to the
standard name as we almost certainly created those.
Reported-by: Petre Rodan <petre.rodan@avira.com>
Signed-off-by: NeilBrown <neilb@suse.de>
One: a single character typo (of instead of or in an error printout)
Two: Audited usage of tfd file descriptor. Make sure that the tfd file
is always closed after usage, and that the tfd variable is reset to -1
if we are going to continue in our loop (not necessary if we know we
will return from our function without going through the dv loop again).
Signed-off-by: Doug Ledford <dledford@redhat.com>
--test can be given in Manage mode.
This can be used when there is an attempt to fail or remove 'faulty',
'failed' or 'detached' devices, or to re-add 'missing' devices.
If no devices were failed, removed, or re-added, then mdadm will
exit with status '2'.
Signed-off-by: NeilBrown <neilb@suse.de>
If the device name "missing" is given for --re-add, then mdadm will
attempt to find any device which should be a member of the array but
currently isn't and will --re-add it to the array.
This can be useful if a device disappeared due to a cabling problem,
and was then re-connected.
The appropriate sequence would be
mdadm /dev/mdX --fail detached
mdadm /dev/mdX --remove detached
mdadm /dev/mdX --re-add missing
Signed-off-by: NeilBrown <neilb@suse.de>
When using 0.90 metadata, devices can be renumbered when
earlier devices are removed.
So when iterating all devices looking for 'failed' or 'detached'
devices, we need to re-check the same slot we checked last time
to see if maybe it has a different device now.
Reported-by: Jim Paris <jim@jtan.com>
Resolves-Debian-Bug: 587550
Signed-off-by: NeilBrown <neilb@suse.de>
This can be used for hot-unplug. When a device has been remove,
udev can call
mdadm --incremental --fail sda
and mdadm will find the array holding sda and remove sda from
the array.
Based on code from Doug Ledford <dledford@redhat.com>
Signed-off-by: NeilBrown <neilb@suse.de>
Allow kernel names like "sda" and "hdb1" to be used to
fail/remove devices from an array.
This is useful as after a device has been removed it can be difficult
to get the major/minor number.
Signed-off-by: NeilBrown <neilb@suse.de>
Allow the name of the array stored in the metadata to be updated. In
some cases the metadata format may not be able to support this rename
without modifying the UUID. In these cases the request will be blocked.
Otherwise we allow the rename to take place, even for active arrays.
This assumes that the user understands the difference between the kernel
node name, the device node symlink name, and the metadata specific name.
Anticipating further need to modify subarrays in-place, introduce the
->update_subarray() superswitch method. A future potential use
case is setting storage pool (spare-group) identifiers.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
mdadm prevents creation when device names are duplicated on the command
line, but leaves the partially created array intact. Detect this case
in the error code from add_to_super() and cleanup the partially created
array. The imsm handler is updated to report this conflict in
add_to_super_imsm_volume().
Note that since neither mdmon, nor userspace for that matter, ever saw an
active array we only need to perform a subset of the cleanup actions.
So call ioctl(STOP_ARRAY) directly and arrange for Create() to cleanup
the map file rather than calling Manage_runstop().
Reported-by: Krzysztof Wojcik <krzysztof.wojcik@intel.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
If /dev is static, a name may remain there after the
device has been detached from the system.
Using 'mdadm' to remove such a device from the array
should still work (even though "mdadm --remove detached"
might be preferred).
So when processing a device for '-r', don't insist on
being able to open the device.
Signed-off-by: NeilBrown <neilb@suse.de>
Minimal changes needed to permit reassembling partially recovered
external metadata arrays. The biggest logical change is that
->container_content() can now surface partially rebuilt members rather
than omitting them from the disk list.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
If a device is marked as faulty, then a re-add will cause it to be
added as a faulty drive, which is not what it wanted.
So just refuse to try to re-add a device which is marked 'faulty'.
Signed-off-by: NeilBrown <neilb@suse.de>
As --add can destroy important data on a disk, and
--re-add is not suppose to, it is wrong to silently
try --add if --re-add fails.
So print a message and abort instead.
Signed-off-by: NeilBrown <neilb@suse.de>
As mdadm is normally a short-lived program it isn't always necessary
to free memory that was allocated, as the 'exit()' call will
automatically free everything. But it is more obviously correct if
the 'free' is there.
So this patch add a few calls to 'free'
Signed-off-by: NeilBrown <neilb@suse.de>
Bother Manage_reconfig and Grow_reshape provide for changing
the 'layout' of a faulty array. This is no necessary.
So discard Manage_reconfig and just use Grow_reshape
Signed-off-by: NeilBrown <neilb@suse.de>
The code for moving spares around a spare-group currently
only works for 0.90 metadata. Generalise it for 1.x metadata
as well.
Reported-by: "Garth Snyder" <garth@grsweb.us>
Signed-off-by NeilBrown <neilb@suse.de>
When building container members with -IR, we need to ensure that
devices added to an active array preserve the 'in_sync' status so they
don't needlessly get rebuilt.
So allow sysfs_add_disk to do this (only works in kernels since
2.6.30) and pass the relevant flag down.
Signed-off-by: NeilBrown <neilb@suse.de>
Skip the unique holder check in the detached case... pretty sure no one is
holding on to it if open() returns ENXIO.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Make sure every failure from add_to_super prints a suitable
error message, and then don't print any error in the caller.
Signed-off-by: NeilBrown <neilb@suse.de>
Not all kernels automatically discard partitions when the
array is stopped, so call the RRPART ioctl to force it.
Signed-off-by: NeilBrown <neilb@suse.de>
This only applies if udev isn't installed or is disabled
by MDADM_NO_UDEV
We try to remove partitions too.
We find names to remove by looking in /var/run/mdadm/map
Signed-off-by: NeilBrown <neilb@suse.de>
When a 'container' gets started, we need udev to notice, but the
kernel has no way of knowing that a KOBJ_CHANGE event is needed. So
send one directly via the 'uevent' sysfs attribute.
Also, uevents don't get generated when md arrays are stopped (prior to
2.6.28) so send 'change' events then too.
Signed-off-by: NeilBrown <neilb@suse.de>
Previously it was possible to set the WRITEMOSTLY flag when
adding a device to an array, but not to clear the flag when re-adding.
This is now possible with --readwrite.
Signed-off-by: NeilBrown <neilb@suse.de>
When adding a device to an array, we check that it is large enough.
Currently the check makes sure there is also room for a reasonably
sized bitmap. But if the array doesn't have a bitmap, then this test
might be too restrictive.
So when adding, only insist there is enough space for the current
bitmap.
When Creating, still require room for the standard sized bitmap.
This resolved Debian Bug 500309
It is currently possible to remove a device and re-add it without the
manager noticing, i.e. without detecting a mdstat->devcnt
container->devcnt mismatch. Introduce ping_manager() to arrange for
mdmon to run manage_container() prior to mdadm dropping the exclusive
open() on the container. Despite these precautions sysfs_read() may
still fail. If this happens invalidate container->devcnt to ensure
manage_container() runs at the next event.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
The action we are waiting for may not be complete until the monitor has
had a chance to take action on the result.
The following script can now remove the device on the first attempt,
versus a few attempts with the original Wait():
#!/bin/bash
#export MDADM_NO_MDMON=1
export IMSM_DEVNAME_AS_SERIAL=1
./mdadm -Ss
./mdadm --zero-superblock /dev/loop[0-3]
echo 2 > /proc/sys/dev/raid/speed_limit_max
./mdadm --create /dev/imsm /dev/loop[0-3] -n 4 -e imsm -a md
./mdadm --create /dev/md/r1 /dev/loop[0-3] -n 4 -l 5 --force -a mdp
./mdadm --fail /dev/md/r1 /dev/loop3
./mdadm --wait /dev/md/r1
x=0
while ! ./mdadm --remove /dev/imsm /dev/loop3 > /dev/null 2>&1
do
x=$((x+1))
done
echo "removed after $x attempts"
./mdadm --add /dev/imsm /dev/loop3
Include 2 small cleanups:
* remove the almost open coded fd2devnum() in Wait() by introducing a
new utility routine stat2devnum()
* teach connect_monitor() to parse the container device from a subarray
string
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
If the metadata_version is
-mdXXX/whatever
rather than
/mdXXX/whatever
then the array is readonly and should be left alone by mdmon.
Signed-off-by: NeilBrown <neilb@suse.de>
We are about to change the syntax of the version string
for 'subarray's. So factor out the test into a single function.
Signed-off-by: NeilBrown <neilb@suse.de>
Adding a device updates the container and then mdmon takes action upon
noticing a change in devices. This reuses the container version of
add_to_super to create a new record for the device.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
If an array is being managed by mdmon, then just
write "inactive" to stop it, and let mdmon do the
final "clear". This makes sure mdmon has a chance
to read the final state and update the metadata properly.
After writing "inactive" with use "ping_monitor" to synchronise
with mdadm, then STOP the array just in case it is still running,
else we will get into an infinite loop in "mdadm -Ss".
Signed-off-by: Neil Brown <neilb@suse.de>
Using buffered IO risks non-atomic updates to parts of the
device that we don't actually want to write to. This isn't in
general safe.
So switch to O_DIRECT for all that IO and make sure we have
properly aligned buffers.
This really should be done in mdadm, not mdmon.
We ensure the device won't be suddenly commited as a hot-spare
using O_EXCL, then check the 'holders' sysfs directory
to make sure it is only in use once.
When loading the metadata for a subarray (super_by_fd), we set
->subarray to be the name read from md/metadata_version so that
getinfo_super can return info about the correct array.
With this we can differentiate between a container and
an array within the container by looking at ->subarray[0].
From: Dan Williams <dan.j.williams@intel.com>
1/ Block attempts to add/remove devices from container members
2/ Forward add/remove requests to containers
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
The current model for creating arrays involves writing
a superblock to each device in the array.
With containers (as with DDF), that model doesn't work.
Every device in the container may need to be updated
for an array made from just some the devices in a container.
So instead of calling write_init_super for each device,
we call it once for the array and have it iterate over
all the devices in the array.
To help with this, ->add_to_super now passes in an 'fd' and name for
the device. These get saved for use by write_init_super. So
add_to_super takes ownership of the fd, and write_init_super will
close it.
This information is stored in the new 'info' field of supertype.
As part of this, write_init_super now removes any old traces of raid
metadata rather than doing this in common code.
array.size is 32bits and counts K. So for arrays with
more than 4Terrabytes, it can overflow.
The correct number can be read from sysfs, but there are still
a few places that use array.size and risk truncation. What is worse.
they compare a number of kilobytes with a number of sectors !!
So use get_component_size() to read the sysfs information, and be
more consistent about units.
Fixes Debian Bug 442874
When we discover that we can 're-add' a drive, we forget to check the
write-mostly flag.
This highlights the fact that you cannot turn 'off' the write-mostly
flag at this point. I wonder if that is a problem...
From: Luca Berra <bluca@comedia.it>
- Fix a bug where mdassemble didn't close a filedescriptor and so couldn't assembele
stacked arrays.
- Allow mdassemble, when run a second time, to mark all arrays as writable.
This is useful if they are started read-only as is best at boot-time.
In 2.6.17 (and prior), the dev_number is ignored when a device
is added to an active array. Rather the first free number is used.
So we work around this by making sure we use the first free
number for dev_number.
Description...
The following patch makes it possible to tag a device as write-mostly on
--add and --re-add with a non-persistent superblock array. Previously,
this was not working.
Signed-Off-By: Paul Clements <paul.clements@steeleye.com>
Signed-off-by: Neil Brown <neilb@suse.de>
As spared don't have a position in the raid array with verion-1 superblocks,
we need to handle them a bit differently.
Signed-off-by: Neil Brown <neilb@suse.de>
Support "--build"ing arrays with bitmaps.
hot-removal of bitmaps
--re-add of drives recently removed.
assorted extra tests
Signed-off-by: Neil Brown <neilb@cse.unsw.edu.au>
Currently this includes
--write-behind to set level of write-behind supported
--write-mostly to flag devices as write-mostly.
Signed-off-by: Neil Brown <neilb@cse.unsw.edu.au>
This allows for larger device number if glibc supports
it (requires 2.3.3).
Also fail before creating larger device number if glibc
support isn't present.
Signed-off-by: Neil Brown <neilb@cse.unsw.edu.au>
Jes Sorensen
Doug Ledford
NeilBrown
Adam Kwolek
Krzysztof Wojcik
Czarnowska, Anna
Dan Williams
Luca Berra