Cloud Computing mit OpenVZ und DRDB

In diesem Projekt geht es darum eine vielzahl verschiedener Server auf weniger Hardware Unterzubringen. Bei austausch Defekter Hardware oder Upgrade soll es zu keinerlei Ausfallzeit kommmen. Aufall soll es nur beim kompletten Versagen eines Gerätes geben.

Software:

  • drbd
  • openvz
  • gfs2
  • *denk*

Hardware:

  • 2x Dell poweregge 2650

Eigentlich geht alles bis jetzt ziemlich schnell, aber ich will euch mal mein Weg nicht vorenthalten:

ISO: debian-504-i386-xfce+lxde-CD-1.iso
Boot menu

  • LXDE
    • Advanced options
      • Expert install

Install Menu

  • Choose Language
    • Deutsch
      • de_DE.UTF-8
    • Locales Menu
      • de_DE
      • de_DE@euro
      • en_US.UTF-8
      • en_US
      • en_US.ISO-8859-15
    • Wählen Sie Ihre Tastaturbelegung aus
      • PC-Tastatur (AT- oder PS/2-Stecker)
      • Deutsch
    • CD-ROM erkennen und einbinden
    • usb-storage Nicht laden ( auser bei einem usb-cdrom laufwerk)
    • PC-Card-Dienst Nicht Starten

  • Installer-Komponenten von CD laden
    • cfdisk-udeb
    • choose-mirror
    • evtl ipv6
    • Network-console
    • parted-udeb
    • evtl crypto*

  • Netzwerk-Hardware erkennen
    • usb-storage Nicht laden (WTF!)
    • PC-Card-Dienst Nicht Starten

  • Netzwerk Einrichten
    • eth0
    • Kein DHCP
    • IP 172.16.200.83
    • Netzmaske: 255.255.0.0
    • Gateway: 172.16.200.10
    • DNS: 172.16.200.10
    • Ja ja alles richtig LMAA2
    • Rechnername: ha0X
    • Domainname: gamenoc.de

  • Installation ueber Fernzugriff (SSH) fortsetzen (grade habe ich mich an den Lärm gewöhnt :-/)
  • Passwort Eingeben
  • Raumwechsel!
  • ssh installer@172.16.200.8
  • Start installer


  • Configure the clock
    • Use NTP
    • 0.debian.pool.ntp.org

  • Detect disks
    • usb-storage Nicht laden ( auser bei einem HDD laufwerk! das will man aber nicht!)

  • Partition disks
    • Guided - use entire disk
    • SCSI2 (0,0,0) (sda) - 36.4 GB DELL PERC Volume
    • All files in one partition (recommended for new users)
    • Finish partitioning and write changes to disk
    • Write the changes to disks? - YES

  • Install the base system
    • linux-image-2.6-686
    • targeted: only include drivers needed for this system

  • Set up users and passwords
    • Enable shadow passwords? - YES
    • Allow login as root? - Yes (Kann man nacher immer noch sperren)
    • passwort
    • Create a normal user account now? - Yes
    • Jan-phil…
    • Login
    • Passwort

  • Configure the package manager
    • Use a network mirror? - YES
    • Services to use - YES

  • Select and install software
    • An der Paketverwendungserfassung teilnehmen? - MM joa muss jeder selber wissen
    • Wir brauchen Keine Packete. einfach Minimal insterlliern (Nix auswaehlen)

  • Install the GRUB boot loader on a hard disk
    • Install GRUB 2 instead of GRUB Legacy? - YES, WIr koennen ja auch grosse Platten Haben
    • Install the GRUB boot loader to the master boot record? - YES auf jedenfall

  • Finish Installation
  • Is the system clock set to UTC? - YES
  • Finish the installation - Fertig werden!
vi /etc/network/interfaces
ha02# diff -u interfaces interfaces.bak
--- interfaces  2010-02-19 11:54:43.000000000 +0100
+++ interfaces.bak      2010-02-19 11:54:09.000000000 +0100
@@ -6,8 +6,7 @@
iface lo inet loopback
 
 # The primary network interface
-auto eth0
-auto eth1
+allow-hotplug eth0
 iface eth0 inet static
        address 172.16.200.83
        netmask 255.255.0.0
@@ -17,8 +16,3 @@
        # dns-* options are implemented by the resolvconf package, if installed
        dns-nameservers 172.16.200.10
        dns-search gamenoc.de
-iface eth1 inet static
-        address 192.168.23.2
-        netmask 255.255.255.252
-        network 192.168.23.0
-        broadcast  192.168.23.3
vi /etc/hosts
ha02# diff -u hosts hosts.bak hosts
--- hosts       2010-02-19 11:50:14.000000000 +0100
+++ hosts.bak   2010-02-19 11:48:01.000000000 +0100
@@ -1,8 +1,5 @@
 127.0.0.1      localhost
-172.16.200.83  ha02-ext.gamenoc.de ha02-ext
-172.16.200.82   ha01-ext.gamenoc.de ha01-ext
-192.168.23.1   ha01 ha01.gamenoc.de
-192.168.23.2   ha02 ha02.gamenoc.de
+172.16.200.83  ha02.gamenoc.de ha02
 
 # The following lines are desirable for IPv6 capable hosts
 ::1     localhost ip6-localhost ip6-loopback
vi /etc/apt/source.list
ha02# diff -u sources.list sources.list.bak
--- sources.list        2010-02-19 11:42:01.000000000 +0100
+++ sources.list.bak    2010-02-19 11:41:51.000000000 +0100
@@ -1,6 +1,7 @@
 # 
 # deb cdrom:[Debian GNU/Linux 5.0.4 _Lenny_ - Official i386 xfce+lxde-CD Binary-1 20100131-19:57]/ lenny main
 
+deb cdrom:[Debian GNU/Linux 5.0.4 _Lenny_ - Official i386 xfce+lxde-CD Binary-1 20100131-19:57]/ lenny main
 
 deb http://mirror.giantix-server.de/debian/ lenny main non-free contrib
 deb-src http://mirror.giantix-server.de/debian/ lenny main non-free contrib

Netzwerk neustarten

apt-get install screen
screen /etc/init.d/networking restart

SSH Keys um configs Schnell auszutauschen

ha01:/home/progamler# ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/root/.ssh/id_dsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_dsa.
Your public key has been saved in /root/.ssh/id_dsa.pub.
The key fingerprint is:
aa:49:ed:67:88:a2:31:0d:96:9d:ed:fe:4c:cd:1a:d3 root@ha01
The key's randomart image is:
+--[ DSA 1024]----+
|                 |
|                 |
|                 |
|  o o            |
|.o o .  S        |
|.o  .. .+        |
|o . .oo= E       |
| o..o++ *        |
|.. .o.o*         |
+-----------------+
ssh-copy-id -i ~/.ssh/id_dsa.pub root@172.16.200.82

Host datei auf ha01 anpassen

scp /etc/hosts ha01:/etc/hosts
apt-get install drbd8-utils drbd8-modules-2.6.26-2-openvz-686 linux-image-2.6.26-2-openvz-686 openssh-server screen gfs2-tools redhat-cluster-modules-2.6-openvz-686 cman

drbd.conf

drdb.conf
global {
        usage-count no;
}
resource r0 {
        protocol C;
        startup {
                wfc-timeout 20;
                degr-wfc-timeout 10;
                become-primary-on both; # Enable this *after* initial testing
        }
 
        syncer {
        rate 125M;
        }
        on ha01 {
        device /dev/drbd0;
        disk /dev/sdb;
        address 192.168.23.1:7788;
        flexible-meta-disk internal; 
        }
        on ha02 {
        device  /dev/drbd0;
        disk /dev/sdb;
        address 192.168.23.2:7788;
        meta-disk internal;
        }
        net {
        allow-two-primaries;
        }
}

Drbd Metadaten auf ha01 & ha02 Erstellen

drbdadm create-md r0
/etc/init.d/drbd restart

ha01 zum primary machen

drbdsetup /dev/drbd0 primary -o 

AUf Syncronisation warten

watch -n 1 cat /proc/drbd
Every 1,0s: cat /proc/drbd                                                          Fri Feb 19 12:35:16 2010

version: 8.0.14 (api:86/proto:86)
GIT-hash: bb447522fc9a87d0069b7e14f0234911ebdab0f7 build by phil@fat-tyre, 2008-11-12 16:40:33
 0: cs:SyncSource st:Primary/Secondary ds:UpToDate/Inconsistent C r---
    ns:6137664 nr:0 dw:0 dr:6143776 al:0 bm:373 lo:66 pe:116 ua:255 ap:0
        [==>.................] sync'ed: 17.3% (28721/34711)M
        finish: 0:15:42 speed: 31,144 (32,452) K/sec
        resync: used:2/61 hits:383367 misses:376 starving:0 dirty:0 changed:376
        act_log: used:0/127 hits:0 misses:0 starving:0 dirty:0 changed:0

/etc/cluster/cluster.conf

cluster.conf
<?xml version="1.0"?>
<cluster alias="openvz" config_version="42" name="openvz">
        <cman expected_votes="1" two_node="1"/>
        <clusternodes>
                <clusternode name="ha01" nodeid="1" votes="1">
                        <fence>
                                <method name="1">
                                        <device domain="ha01" name="xvm"/>
                                </method>
                        </fence>
                </clusternode>
                <clusternode name="ha02" nodeid="2" votes="1">
                        <fence>
                                <method name="1">
                                        <device domain="ha02" name="xvm"/>
                                </method>
                        </fence>
                </clusternode>
        </clusternodes>
        <fencedevices>
                <fencedevice agent="fence_xvm" name="xvm"/>
        </fencedevices>
        <rm/>
</cluster>
scp /etc/cluster/cluster.conf ha02:/etc/cluster/cluster.conf
/etc/init.d/cman start
gfs_mkfs -t openvz:data -p lock_dlm -j 2 /dev/drbd0
mkdir /vserver
mount /dev/drbd0 /vserver/
cd /vserver
mkdir ha01
mkdir ha02
ha02:/var/lib/vz# mv * /vserver/ha02/
ln -s /vserver/ha02/* .
mv /etc/vz/conf/ /vserver/ha02/
ln -s /vserver/ha02/conf/ .
mv /vserver/ha01/template/ /vserver/
ln -s /vserver/template/ /vserver/ha02/
mv /etc/rc2.d/S20vz /etc/rc2.d/S99vz
mv /etc/rc2.d/S70drbd /etc/rc2.d/S20drbd
mv /etc/rc3.d/S20vz /etc/rc2.d/S99vz
mv /etc/rc3.d/S70drbd /etc/rc2.d/S20drbd
mv /etc/rc4.d/S20vz /etc/rc2.d/S99vz
mv /etc/rc4.d/S70drbd /etc/rc2.d/S20drbd
mv /etc/rc5.d/S20vz /etc/rc2.d/S99vz
mv /etc/rc5.d/S70drbd /etc/rc2.d/S20drbd
ha01:/var/lib/vz# mv * /vserver/ha01/
ln -s /vserver/ha01/* .
mv /etc/vz/conf/ /vserver/ha01/
ln -s /vserver/ha01/conf/ .
rm /vserver/ha01/template/ -r
ln -s /vserver/template/ /vserver/ha01/
mv /etc/rc2.d/S20vz /etc/rc2.d/S99vz
mv /etc/rc2.d/S70drbd /etc/rc2.d/S20drbd
mv /etc/rc3.d/S20vz /etc/rc2.d/S99vz
mv /etc/rc3.d/S70drbd /etc/rc2.d/S20drbd
mv /etc/rc4.d/S20vz /etc/rc2.d/S99vz
mv /etc/rc4.d/S70drbd /etc/rc2.d/S20drbd
mv /etc/rc5.d/S20vz /etc/rc2.d/S99vz
mv /etc/rc5.d/S70drbd /etc/rc2.d/S20drbd

/etc/fstab

fstab
# /etc/fstab: static file system information.
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
proc            /proc           proc    defaults        0       0
/dev/sda1       /               ext3    errors=remount-ro 0       1
/dev/sda5       none            swap    sw              0       0
/dev/hda        /media/cdrom0   udf,iso9660 user,noauto     0       0
/dev/fd0        /media/floppy0  auto    rw,user,noauto  0       0
/dev/drbd0      /vserver    auto   defaults 0 0
vi /etc/init.d/drbdp
#!/bin/bash
ST=$( drbdadm dstate r0 2> /dev/null)
ST=${ST%/*}

echo $ST
while [ $ST != "UpToDate" ] ; do
        ST=$( drbdadm dstate r0 2> /dev/null)
        ST=${ST%/*}
done
drbdadm primary all
mount -a

ha02

chmod 777 /etc/init.d/drbdp
scp /etc/init.d/drbdp ha01:/etc/init.d/drbdp
update-rc.d drbdp defaults 21

ha01

chmod 777 /etc/init.d/drbdp
update-rc.d drbdp defaults 21
ha01:/vserver/template/cache# wget http://download.openvz.org/template/precreated/debian-5.0-x86.tar.gz
  • Zuletzt geändert: 11.07.2016 23:18
  • (Externe Bearbeitung)