PPTV
Download
1 / 45

PPTV 分布式装机系统 - PowerPoint PPT Presentation


  • 217 Views
  • Uploaded on

PPTV 分布式装机系统. KenZhu [email protected] 10/9 2012. Agenda. PPTV 分布式装机系统介绍 实际面临的问题 解决方案 系统部署 系统组件介绍 系统实现 Cobbler 配置文件同步 Cobbler 自动部署 应用实例 Cobbler Quick-Start Cobbler 动态 ks file Cobbler Tips 二 次开发. 实际面临的问题. 服务器系统安装处于半手工状态,有些 isp 机房甚至需要现场通过光盘安装,耗费大量人力和时间

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' PPTV 分布式装机系统' - jock


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

PPTV分布式装机系统

KenZhu

[email protected]

10/9 2012


Agenda
Agenda

  • PPTV分布式装机系统介绍

    • 实际面临的问题

    • 解决方案

    • 系统部署

    • 系统组件介绍

  • 系统实现

    • Cobbler

    • 配置文件同步

    • Cobbler 自动部署

  • 应用实例

    • Cobbler Quick-Start

    • Cobbler 动态ks file

    • Cobbler Tips

  • 二次开发


实际面临的问题

  • 服务器系统安装处于半手工状态,有些isp机房甚至需要现场通过光盘安装,耗费大量人力和时间

  • 非标准化的系统安装,导致业务部署时遇到这样那样的奇怪问题

  • 系统安装是瓶颈,由于以上问题导致业务需要快速部署时,无机器可用的现象


实际面临的问题

  • 众多机房,由于网络互通互联的问题,只在核心机房架设kickstart server完全无法解决问题;如采用分布式架构,每个机房部署一台kickstart server;如何统一配置,安装模板以及安装源等

  • 业务众多,每个业务对应静态安装模板,不便于管理

  • 对于某些机房没有可以独立使用机器,在业务机如何复用的问题


解决方案

  • Cobbler 开源项目;基于kickstart,方便快速的网络安装操作系统

  • Git分布式版本控制;静态配置文件的同步

  • Django框架;动态配置文件的同步

  • Rsyncd;安装源同步

  • 编译Cobbler依赖服务apache, python并打包;解决复用的问题

  • 使用ControlTie自动部署



系统部署

  • BGP机房搭建一台Cobbler Master;该服务器管理系统的配置,安装模板以及cobbler部署包;并同时作为该机房的cobbler server提供安装服务

  • 所有核心机房部署Cobbler Secondary;作为本机房的安装服务器并提供安装源同步

  • 其他机房的Cobbler,由ControlTie自动部署,可以复用业务机器



系统组件

  • Cobbler 主要服务,提供kickstart server

  • Git Repository 静态配置文件同步源

  • Gitgit客户端,用于同步源上的配置文件

  • Django调用cobbler API, 同步动态配置文件

  • Nginx发布Git repository & Djangoapplication

  • Rsync用于安装源同步

  • Python 包含cobbler依赖模块的python

  • Apache 发布cobbler安装源


系统实现

  • Cobbler

    • Cobbler 介绍

    • Cobbler models

    • Cobbler 动态KS

  • 配置文件同步

    • Git静态配置文件同步

    • Django动态配置文件同步

    • 配置文件同步脚本

  • Cobbler 自动部署

    • Cobbler部署包

    • ControlTie部署


Cobbler
Cobbler

  • 基于kickstart, linux系统网络快速安装

  • 按不同对象组合动态生成kickstart template, 通过ksmeta, snippets实现

  • Koan方便实现重装系统,安装虚拟机

  • 统一管理dhcp, tftp, dns这些网络安装依赖的服务

  • 支持大部分主流linux发行版本,RHEL, Debian, Suse, Ubuntu, Fedora etc.



Cobbler models1
Cobbler Models

  • Distro定义安装源,kernel, initrd

  • Profile 定义KSfile等信息,可以认为该对象是对应一个业务的template

  • System 定义系统信息,比如hostname, ip地址等信息


Cobbler ks
Cobbler 动态KS

  • KS file 使用snippet模块化,便于管理

  • 通过cobbler models中不同参数,动态生成标准化的ksfile

  • Cheetah实现模板中变量替换,以及逻辑的运算;其中的伪代码通过python编译器解析


Git静态配置文件同步

  • 所谓静态配置文件是指每台cobblerserver上基本一致的配置文件,比如kickstartfiles, snippets 等

  • Cobbler Master上建立git repository

  • Cobbler部署时通过git clone同步kickstarts, snippets, triggers 这些静态配置文件

  • Crontab定期从Master那里git pull回最新的配置


Git

  • 目前流行的版本控制工具

  • 分布式版本控制软件

  • 对比集中式,分布式提供离线版本控制

  • 为应用提供配置文件的版本控制&分布式同步方式


配置文件部署

Git clone生成kickstart, snippets, triggers这些配置文件,并和git repository上的同步

...

test -x /var/lib/cobbler && cd /var/lib/cobbler/ || {

echo -ne "cobbler directory doesn't exist! \n"

exit 0

}

rm -rfkickstarts

rm -rf snippets

rm -rf triggers

git clone http://<cobbler master>/cobbler-git/kickstarts.git

git clone http://<cobbler master>/cobbler-git/snippets.git

git clone http://<cobbler master>/cobbler-git/triggers.git

...


同步脚本

使用git pull命令从git repository拉取配置更新

echo -ne "Kickstarts sync "

cd $CB_DATA_PATH/kickstarts

git pull && echo -ne "[OK]\n" || echo -ne "[FAILED]\n"

echo -ne "Snippets sync "

cd $CB_DATA_PATH/snippets

git pull && echo -ne "[OK]\n" || echo -ne "[FAILED]\n"

echo -ne "Triggers sync "

cd $CB_DATA_PATH/triggers

git pull && echo -ne "[OK]\n" || echo -ne "[FAILED]\n"


Django
Django动态配置文件同步

  • 所谓动态配置文件是指由于不同cobbler配置,而产生的配置文件;比如每台cobbler上的profile对象,由所在cobblerserver的distro对象决定

  • 通过django调用cobbler api

  • 使用nginx发布django应用,可通过http远程调用

  • Crontab定期调用,通过传入参数,获取配置文件列表并同步


Profile
Profile对象查询接口

...

from cobbler.api import BootAPI #cobbler api

#传入distro参数,返回profile配置文件列表

defprofile_list(request, distro):

cb_api = BootAPI()

profiles = cb_api.find_profile(return_list=True, distro=distro)

data = []

for profile in profiles:

profile_url = "%s/%s.json" % (settings.PROFILES_URL, profile.name)

data.append(profile_url)

return HttpResponse("|".join(data), 'text/html')

...


同步脚本

...

DISTROS=$(cobbler distro list) #取得此cobbler的所有distro对象

for distro in $DISTROS

do

PROFILES=$(curl http://<cobbler master>/profile_list/$distro/ 2>/dev/null | tr ‘\|’ ‘ ’) #访问Profile对象查询接口

for profile in $PROFILES

do

echo -ne "Download $profile \t\t"

wget -q $profile && echo -ne "[OK]\n" || echo -ne "[FAILED]\n"

done

done

...


Cobbler1
Cobbler 自动部署

  • Cobbler 部署包

  • ControlTieTask

    • Cobbler 部署

    • Cobbler 配置

    • Cobbler 安装源同步


Cobbler2
Cobbler 部署包

  • 自动部署时包含cobbler源码以及依赖模块,服务的包

  • Cobbler源码

  • Apache2.2.21 编译时必须包含wsgi_module模块

  • Python2.7

  • Cheetah, yaml, pykickstart相应python模块


Ct cobbler
CT - Cobbler 部署

步骤

  • 设置相应环境变量,系统设置(关闭iptables, selinux, yum设置)

  • 通过yum安装所需服务的rpm包

  • 下载 cobbler部署包,并在本机进行编译安装

  • 同步配置文件

  • 部署完成


Ct cobbler1
CT – Cobbler 部署

#install required rpms

yum install createrepodhcp curl-develtftp-server yum-utilsmkisofsgit -y

#fetch cobbler package

cd /tmp

wget http://<cobbler master>/packages/cobbler_proxy-2.2.0.tar.bz2 --output-document=cobbler_proxy-2.2.0.tar.bz2

#cobbler installation

cd /home/tools

tar jxvf /tmp/cobbler_proxy-2.2.0.tar.bz2

cd /home/tools/package; rpm -ihv *.rpm

cd /home/tools/cobbler-2.2.0; make; make install;

grep "tools" /etc/ld.so.conf || sed -i 'a/\/home\/tools\/python\/lib/' /etc/ld.so.conf && echo -ne ""


Ct cobbler2
CT – Cobbler设置

  • 对于cobbler参数进行设置Cobbler server, next_server

  • dhcp参数设置;

  • 这些参数通过ControlTie传入


Ct cobbler3
CT – Cobbler设置

...

COBBLER_SETTING=/etc/cobbler/settings

DHCP_SETTING=/etc/cobbler/dhcp.template

#Cobbler settings

sed -i '/^server:/ cserver: ${option.server_ip}' $COBBLER_SETTING

sed -i '/^next_server:/ cnext_server: ${option.server_ip}' $COBBLER_SETTING

#DHCP settings

sed -i '/^subnet.*/ csubnet ${option.netid} netmask ${option.netmask} {' $DHCP_SETTING

sed -i '/option domain-name-servers/ coption domain-name-servers 8.8.8.8;' $DHCP_SETTING

sed -i '/option subnet-mask/ coption subnet-mask ${option.netmask};' $DHCP_SETTING

sed -i '/range dynamic-bootp/ crange dynamic-bootp ${option.pool_range};' $DHCP_SETTING

sed -i '/option routers/ coption routers ${option.gateway};' $DHCP_SETTING

...


CT – 安装源同步

  • 选择ISP和所需同步的系统版本


CT – 安装源同步

if [ ${option.isp} = "C1" ]

then

case ${option.release} in

"5.4")

echo -ne "sync 5.4 from T1 \n"

cobbler import --path=rsync://<cobbler master>/RHEL5.4 --name=RHEL_Server_5.4_PPLive --arch=x86_64

;;

"5.8")

echo -ne "sync 5.8 from T1 \n"

cobbler import --path=rsync://<cobbler master>/RHEL5.8 --name=RHEL_Server_5.8_PPLive --arch=x86_64

;;

esac


应用实例

  • Cobbler Quick-Start

    • 物理机安装

    • 物理机重装

    • 虚拟机安装

  • Cobbler 动态ks file实例

    • YUM configuration snippet

  • Cobbler tips


Machine without os
Machine without OS

  • Access idrac interface with web browser J2EE installed

  • Start remote console

  • Reboot or startup machine via idrac

  • Press “F12” to enter PXE mode when machine booting

  • Choose 1 profile you want to apply on this machine in cobbler menu

  • Starting unattended installation

  • Configure network interface on the machine when installation finished



Machine with os
Machine with OS

  • Use koan to reinstall

    yum install koan –y

  • Create a new system on cobbler server

    cobbler system add --name=JA03469D1201 --profile=cob_5.4_base --ip-address=<ip address> --gateway=<gwip> --static=true --subnet=255.255.0.0

  • Execute koan on this machine to replace itself

    koan --server=<cobbler server> --system=JA03469D1201 --replace-self


Xen virtual machine installation
Xen Virtual-machine installation

Cobbler server: <cobbler server ip>

  • 首先在cmdb中通过IP确认宿主机资产编号,然后生成虚拟机资产编号,并在cobbler server生成system信息。以<host>这台机器为例,宿主机为<xen host>,资产编号JA03469D1201,由于该机器为D1,故该虚拟机资产编号为JA03469D1201-1,cobbler system name和资产编号一致。

  • 在cobbler server输入如下命令,添加system

    cobbler system add --name=JA03469D1201-1 --profile=vs_5.4 --ip-address=<ip> --gateway=<gwip> --static=true –subnet =255.255.255.0

  • 安装koan

    Yum install koan –y


Xen virtual machine installation1
Xen Virtual-machine installation

  • 在宿主机上,执行命令安装虚拟机

    koan --server=<cobbler server> --virt --system=JA03469D1201-1 --virt-name=D1 --virt-path=/dev/sdb --virt-bridge=xenbr0 -=–nogfx

  • server: cobbler server

  • virt-system: system name you appended before

  • virt-name: xen virtual server name

  • virt-path: disk device

  • virt-bridge: network bridge


Ks file
动态ksfile 实例

RHEL5.4 KS file - distro:RHEL5.4 profile:vs_5.4

#[SNIPPET] yum_configuration

$SNIPPET('pptv/yum_configuration')

RHEL5.8 KS file - distro:RHEL5.8 profile:vs_5.8

#[SNIPPET] yum_configuration

$SNIPPET('pptv/yum_configuration')


Snippet yum confiuration
Snippet/yum_confiuration

#set distro=$getVar(‘distro_name’,‘’) 定义distro变量,该变量值为该机器所属distro的name

#if ‘5.4’ in $distro判断是否distro为5.4,为真则写入相应配置

##Start of YUM Configuration

##End of YUM Configuration

cat > /etc/yum.repos.d/PPLive.repo << "EOF"

[RHEL 5.4 YUM source]

EOF

#else if '5.8' in $distro判断是否distro为5.8,为真则写入相应配置

##Start of YUM Configuration

cat > /etc/yum.repos.d/PPLive.repo << "EOF"

[RHEL 5.8 YUM source]

EOF

#else if '6.2' in $distro

##Start of 6.2 YUM Configuration

cat > /etc/yum.repos.d/PPLive.repo << "EOF"

[RHEL 6.2 YUM source]

EOF

##End of 6.2 YUM Configuration

#end if


Ks file1
生成ks file

  • 访问CobblerServer的相应URL,可以验证生成的ks file

  • Profile:/cblr/svc/op/ks/profile/<profile name>

  • System: :/cblr/svc/op/ks/system/<system name>


Cobbler tips dhcp
Cobbler tips – dhcp设置

subnet 10.x.0.0 netmask 255.255.0.0 {

pool {

allow members of "pxe";

allow members of "etherboot";

allow members of "anaconda";

option routers 10.x.1.1;

option domain-name-servers 8.8.8.8;

option subnet-mask 255.255.0.0;

range dynamic-bootp10.x.250.2 10.x.250.100;

filename "/pxelinux.0";

default-lease-time 600;

max-lease-time 600;

next-server x.x.x.x;

}

}


Base os kernel options
Base OS kernel options

# kernel options that should be present in every cobbler installation.

# kernel options can also be applied at the distro/profile/system

# level.

kernel_options:

ksdevice: bootif

dhcpclass: Anaconda #对应dchp中的tag设置

lang: 'en_US'

text: ~

sshd: ~

vnc: ~

vncpassword: 'cobblervnc'

noipv6: ~


次开发

  • Cobbler API

  • Cobbler xmlrpc远程调用

  • PPTV虚拟机自动安装应用


Cobbler api
Cobbler API

  • 通过Cobbler.api模块可以操作cobbler里所有对象

  • 研究cli.py这个命令接口模块,可以很快了解API的使用


Cobbler xmlrpc
Cobbler xmlrpc远程调用

  • 默认cobbler xmlrpc服务绑定在localhost:25151

  • 通过修改utils.py模块,可以绑定在外部借口,远程进行cobbler对象的操作

  • 也可通过方向代理工具,转发至该端口;对外开放该服务


PPTV 虚拟机自动安装应用