建设一套企业 DevOps + 协同办公 + 可观测性 + 网络基础设施平台，并希望通过：
- Ansible Playbook：负责服务器初始化、软件部署、配置管理
- Docker Compose：负责应用容器编排
实现一键化部署
所有服务不在一台机器，而是拆成多个角色节点
最佳设计不是一个 site.yml 一次性全部安装，而是设计成：
模块化 Ansible + Docker Compose 平台
特点：
✅ 每个服务独立安装
✅ 每个服务独立卸载
✅ 每个服务独立升级
✅ 支持安装前依赖检查
✅ 支持多节点部署
✅ 支持生产环境 HA
✅ 支持 Docker Compose 服务
✅ 支持 Kubernetes/Helm 服务
✅ 支持统一配置管理
一、最终项目结构
enterprise-platform-ansible/
├── README.md
├── ansible.cfg
│
├── inventory/
│
│ ├── production/
│ │
│ │ ├── hosts.ini
│ │ │
│ │ └── group_vars/
│ │ ├── all.yml
│ │ ├── database.yml
│ │ ├── ldap.yml
│ │ ├── docker.yml
│ │ └── kubernetes.yml
│
│
├── playbooks/
│
│ ├── install.yml # 总入口
│ ├── uninstall.yml
│ ├── upgrade.yml
│
│
│ ├── common.yml
│ ├── docker.yml
│ ├── kubernetes.yml
│
│ ├── keepalived.yml
│ ├── haproxy.yml
│
│ ├── dns.yml
│ ├── proxy.yml
│
│ ├── ldap.yml
│ ├── keycloak.yml
│ ├── vault.yml
│
│ ├── gitlab.yml
│ ├── gerrit.yml
│ ├── jenkins.yml
│ ├── nexus.yml
│ ├── sonar.yml
│ ├── harbor.yml
│
│ ├── jira.yml
│ ├── confluence.yml
│ ├── nextcloud.yml
│ ├── collabora.yml
│ ├── onlyoffice.yml
│
│ ├── mail.yml
│ ├── ftp.yml
│ ├── openvpn.yml
│
│ ├── prometheus.yml
│ ├── alertmanager.yml
│ ├── grafana.yml
│ ├── loki.yml
│ ├── elk.yml
│ ├── thanos.yml
│
│ ├── argocd.yml
│ ├── velero.yml
│ └── restic.yml
│
│
├── roles/
│
│ ├── common/
│ ├── docker/
│ ├── kubernetes/
│ ├── keepalived/
│ ├── haproxy/
│ ├── nginx/
│
│ ├── dns/
│ ├── proxy/
│
│ ├── ldap/
│ ├── keycloak/
│ ├── vault/
│
│ ├── gitlab/
│ ├── gerrit/
│ ├── jenkins/
│ ├── nexus/
│ ├── sonar/
│ ├── harbor/
│
│ ├── jira/
│ ├── confluence/
│ ├── nextcloud/
│ ├── collabora/
│ ├── onlyoffice/
│
│ ├── mail/
│ ├── ftp/
│ ├── openvpn/
│
│ ├── prometheus/
│ ├── alertmanager/
│ ├── grafana/
│ ├── loki/
│ ├── elk/
│ ├── thanos/
│
│ ├── argocd/
│ ├── velero/
│ └── restic/
│
│
└── docker-compose/
├── gitlab/
│ └── docker-compose.yml
│
├── gerrit/
│
├── jenkins/
│
├── nexus/
│
├── sonar/
│
├── ldap/
│
├── jira/
│
├── confluence/
│
├── nextcloud/
│
├── mail/
│
├── ftp/
│
└── openvpn/
二、安装方式设计
1. 单独安装 GitLab
执行：
ansible-playbook \
-i inventory/production/hosts.ini \
playbooks/gitlab.yml
只安装：
GitLab
+
PostgreSQL
+
Redis
+
Nginx
2. 单独安装 Jenkins
ansible-playbook \
-i inventory/production/hosts.ini \
playbooks/jenkins.yml
结果：
Docker
└── Jenkins Container
3. 单独安装 Nexus
ansible-playbook \
playbooks/nexus.yml
部署：
Nexus Repository
docker-compose
4. 单独安装 Kubernetes
ansible-playbook \
playbooks/kubernetes.yml
安装：
containerd
kubeadm
kubelet
kubectl
三、Playbook模板
例如：
gitlab.yml
---
- hosts: gitlab
become: yes
roles:
- gitlab
roles/gitlab/tasks/main.yml
---
- name:
create directory
file:
path:
/opt/gitlab
state:
directory
- name:
copy compose file
template:
src:
docker-compose.yml.j2
dest:
/opt/gitlab/docker-compose.yml
- name:
start gitlab
community.docker.docker_compose_v2:
project_src:
/opt/gitlab
四、支持 uninstall
例如：
ansible-playbook \
playbooks/uninstall.yml \
-e service=gitlab
执行：
docker compose down
remove container
remove volume(optional)
五、支持升级
例如：
升级 Jenkins：
ansible-playbook \
playbooks/upgrade.yml \
-e service=jenkins
流程：
backup
pull new image
restart
health check
六、服务依赖关系
自动检查：
例如：
GitLab
依赖：
docker
postgresql
redis
smtp
ldap(optional)
Nextcloud
依赖：
docker
postgres
redis
ldap(optional)
collabora(optional)
onlyoffice(optional)
Grafana
依赖：
prometheus
loki(optional)
ldap(optional)
七、统一变量
group_vars/all.yml
domain:
example.com
timezone:
Asia/Shanghai
docker_root:
/data/docker
storage_root:
/mnt/allin_data
registry:
harbor.example.com
ldap:
dc=example,dc=com
八、服务安装矩阵
| 服务 | 方式 |
|---|---|
| Gerrit | Docker Compose |
| GitLab | Docker Compose |
| Jira | Docker Compose |
| Confluence | Docker Compose |
| DNS | Ansible + Bind9 |
| Proxy | Squid/Nginx |
| Nexus | Docker Compose |
| Jenkins | Docker Compose |
| LDAP | Docker Compose |
| Nextcloud | Docker Compose |
| Collabora | Docker Compose |
| OnlyOffice | Docker Compose |
| Docker Compose | |
| Sonar | Docker Compose |
| ELK | Docker Compose/K8s |
| Loki | K8s Helm |
| Prometheus | K8s Helm |
| Alertmanager | K8s Helm |
| Grafana | K8s Helm |
| Thanos | K8s Helm |
| FTP | Docker Compose |
| OpenVPN | Docker Compose |
| Harbor | Helm |
| Vault | Helm |
| Keycloak | Helm |
| ArgoCD | Helm |
| Velero | Helm |
| Restic | Ansible |
| Keepalived | Ansible |
| HAProxy | Ansible |
九、最终使用方式
安装全部
ansible-playbook \
-i inventory/production/hosts.ini \
playbooks/install.yml
只安装 DevOps
ansible-playbook \
playbooks/gitlab.yml
ansible-playbook \
playbooks/jenkins.yml
ansible-playbook \
playbooks/nexus.yml
ansible-playbook \
playbooks/sonar.yml
只安装监控
ansible-playbook \
playbooks/prometheus.yml
ansible-playbook \
playbooks/grafana.yml
ansible-playbook \
playbooks/loki.yml
十、建议下一步生成顺序
为了最终能直接落地，我建议按以下顺序生成代码：
Part 1 基础层
- ansible.cfg
- inventory
- common role
- Docker role
- Nginx
- Keepalived
- HAProxy
Part 2 身份认证
- LDAP
- Keycloak
- Vault
Part 3 DevOps
- GitLab
- Gerrit
- Jenkins
- Nexus
- Sonar
- Harbor
Part 4 Office
- Jira
- Confluence
- Nextcloud
- Collabora
- OnlyOffice
Part 5 Kubernetes
- kubeadm
- ArgoCD
- Harbor
- Velero
Part 6 Monitoring
- Prometheus
- Alertmanager
- Grafana
- Loki
- ELK
- Thanos
这样最终会得到一个可执行的：
enterprise-platform-ansible/
每个服务均可：
install
uninstall
upgrade
backup
restore
独立管理。
按照十的顺序生成代码
好的，我按照之前规划的 10个阶段 生成代码。
由于完整项目包含：
- 40+ Ansible Role
- 30+ Playbook
- 20+ Docker Compose
- Kubernetes Helm部署
- 配置模板
- 备份恢复脚本
一次消息无法完整放下，会分批输出
生成顺序如下：
Part 1 基础层
├── ansible.cfg
├── inventory
├── group_vars
├── common role
├── Docker role
├── Nginx role
├── Keepalived role
└── HAProxy role
Part 2 身份认证层
├── LDAP
├── Keycloak
└── Vault
Part 3 DevOps层
├── GitLab
├── Gerrit
├── Jenkins
├── Nexus
├── Sonar
└── Harbor
Part 4 Office层
├── Jira
├── Confluence
├── Nextcloud
├── Collabora
├── OnlyOffice
├── Mail
├── FTP
└── OpenVPN
Part 5 Kubernetes层
├── containerd
├── kubeadm
├── Kubernetes Cluster
└── ArgoCD
Part 6 Monitoring层
├── Prometheus
├── Alertmanager
├── Grafana
├── Loki
├── ELK
└── Thanos
Part 7 Backup层
├── Velero
└── Restic
Part 8 安全增强
├── TLS
├── Cert-manager
├── Trivy
└── OPA
Part 9 CI/CD集成
├── GitLab Runner
├── Jenkins Pipeline
├── Harbor CI
└── ArgoCD GitOps
Part 10 运维管理
├── install
├── uninstall
├── upgrade
├── backup
└── restore
Part 1 基础层
目录
创建：
mkdir -p enterprise-platform-ansible
cd enterprise-platform-ansible
mkdir -p \
inventory/production/group_vars \
playbooks \
roles/{common,docker,nginx,keepalived,haproxy}
最终：
enterprise-platform-ansible
├── ansible.cfg
├── inventory
├── playbooks
└── roles
1. ansible.cfg
文件：
ansible.cfg
内容：
[defaults]
inventory = inventory/production/hosts.ini
remote_user = root
host_key_checking = False
forks = 20
timeout = 30
roles_path = roles
[privilege_escalation]
become = true
become_method = sudo
[ssh_connection]
pipelining = True
2. Inventory
文件：
inventory/production/hosts.ini
内容：
################################
# Load Balancer
################################
[loadbalancer]
lb01 ansible_host=10.50.0.10
lb02 ansible_host=10.50.0.11
################################
# Proxy
################################
[proxy]
proxy01 ansible_host=10.50.0.20
################################
# Docker Host
################################
[docker_hosts]
docker01 ansible_host=10.50.0.30
################################
# Kubernetes
################################
[k8s_master]
master01 ansible_host=10.50.0.40
master02 ansible_host=10.50.0.41
master03 ansible_host=10.50.0.42
[k8s_worker]
worker01 ansible_host=10.50.0.50
worker02 ansible_host=10.50.0.51
################################
# Monitoring
################################
[monitor]
monitor01 ansible_host=10.50.0.60
3. 全局变量
文件：
inventory/production/group_vars/all.yml
################################
# System
################################
timezone: Asia/Shanghai
hostname_domain:
company.local
################################
# Storage
################################
data_root:
/data
docker_root:
/data/docker
################################
# Network
################################
vip:
10.50.0.100
interface:
eth0
################################
# Docker
################################
docker_version:
latest
################################
# Kubernetes
################################
kubernetes_version:
"1.31.0"
4. Common Role
目录：
roles/common
创建：
mkdir -p roles/common/tasks
roles/common/tasks/main.yml
---
- name:
update system
apt:
update_cache: yes
- name:
install basic packages
apt:
name:
- vim
- curl
- wget
- net-tools
- jq
- unzip
- chrony
- lvm2
state:
present
- name:
set timezone
timezone:
name:
"{{ timezone }}"
- name:
enable chrony
service:
name:
chrony
state:
started
enabled:
yes
5. Docker Role
目录：
roles/docker
创建：
mkdir -p roles/docker/tasks
roles/docker/tasks/main.yml
---
- name:
install docker
apt:
name:
- docker.io
- docker-compose-plugin
state:
present
- name:
enable docker
systemd:
name:
docker
state:
started
enabled:
yes
6. 基础安装Playbook
文件：
playbooks/bootstrap.yml
内容：
---
- hosts:
all
become:
yes
roles:
- common
- hosts:
docker_hosts
become:
yes
roles:
- docker
执行：
ansible-playbook playbooks/bootstrap.yml
完成：
- 系统初始化
- Docker安装
7. Keepalived Role
目录：
mkdir -p roles/keepalived/{tasks,templates}
tasks/main.yml
---
- name:
install keepalived
apt:
name:
keepalived
state:
present
- name:
config keepalived
template:
src:
keepalived.conf.j2
dest:
/etc/keepalived/keepalived.conf
- name:
restart
systemd:
name:
keepalived
state:
restarted
templates/keepalived.conf.j2
vrrp_instance VI_1 {
state MASTER
interface {{ interface }}
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 123456
}
virtual_ipaddress {
{{ vip }}
}
}
8. HAProxy Role
目录：
roles/haproxy
tasks：
---
- name:
install haproxy
apt:
name:
haproxy
state:
present
- name:
config
template:
src:
haproxy.cfg.j2
dest:
/etc/haproxy/haproxy.cfg
- name:
restart
systemd:
name:
haproxy
state:
restarted
haproxy.cfg.j2
global
daemon
defaults
mode tcp
timeout connect 5s
timeout client 1m
timeout server 1m
frontend https
bind *:443
default_backend backend_apps
backend backend_apps
balance roundrobin
server app01 10.50.0.30:443 check
server app02 10.50.0.31:443 check
9. HA Playbook
文件：
playbooks/ha.yml
---
- hosts:
loadbalancer
roles:
- keepalived
- haproxy
执行：
ansible-playbook playbooks/ha.yml
Part 1 完成
当前已生成：
✅ Ansible基础框架
✅ Inventory
✅ 全局变量
✅ 系统初始化
✅ Docker安装
✅ Keepalived
✅ HAProxy
Part 2 身份认证层
本部分包含：
LDAP Server
├── OpenLDAP
├── phpLDAPadmin
└── LDAP Schema
Keycloak
├── LDAP Federation
├── OIDC
├── OAuth2
└── SSO
Vault
├── Standalone / HA
├── LDAP Authentication
├── Kubernetes Authentication
└── Secret Management
目标：
Keycloak
|
|
OpenLDAP
|
+---------------+---------------+
| | |
GitLab Jenkins Grafana
Jira Nextcloud ArgoCD
Confluence Harbor
Part 2目录增加
最终：
enterprise-platform-ansible/
├── playbooks/
│
├── ldap.yml
├── keycloak.yml
└── vault.yml
├── roles/
├── ldap/
│ ├── tasks/
│ ├── templates/
│ └── defaults/
├── keycloak/
│ ├── tasks/
│ ├── templates/
│ └── defaults/
└── vault/
├── tasks/
├── templates/
└── defaults/
├── docker-compose/
├── ldap/
├── keycloak/
└── vault/
一、LDAP Server
采用：
- OpenLDAP
- phpLDAPadmin
Docker Compose部署。
目录：
mkdir -p docker-compose/ldap
1. LDAP docker-compose
文件：
docker-compose/ldap/docker-compose.yml
services:
openldap:
image: osixia/openldap:latest
container_name: openldap
restart: always
environment:
LDAP_ORGANISATION: "AllinTech"
LDAP_DOMAIN: "company.local"
LDAP_ADMIN_PASSWORD: "Admin@123456"
ports:
- "389:389"
- "636:636"
volumes:
- /data/ldap/database:/var/lib/ldap
- /data/ldap/config:/etc/ldap/slapd.d
phpldapadmin:
image: osixia/phpldapadmin:latest
container_name: phpldapadmin
restart: always
environment:
PHPLDAPADMIN_LDAP_HOSTS: openldap
ports:
- "8088:80"
depends_on:
- openldap
2. LDAP目录规划
服务器：
/data/ldap
├── database
└── config
Ansible创建：
- name:
create ldap directory
file:
path:
/data/ldap
state:
directory
3. LDAP Role
创建：
mkdir -p roles/ldap/{tasks,templates,defaults}
defaults/main.yml
ldap_domain:
company.local
ldap_password:
Admin@123456
ldap_org:
AllinTech
tasks/main.yml
---
- name:
create ldap directory
file:
path:
/opt/ldap
state:
directory
- name:
copy compose
template:
src:
docker-compose.yml.j2
dest:
/opt/ldap/docker-compose.yml
- name:
start ldap
community.docker.docker_compose_v2:
project_src:
/opt/ldap
templates/docker-compose.yml.j2
services:
openldap:
image:
osixia/openldap:latest
restart:
always
environment:
LDAP_DOMAIN:
"{{ ldap_domain }}"
LDAP_ADMIN_PASSWORD:
"{{ ldap_password }}"
ports:
- 389:389
- 636:636
phpldapadmin:
image:
osixia/phpldapadmin
environment:
PHPLDAPADMIN_LDAP_HOSTS:
openldap
ports:
- 8088:80
4. LDAP Playbook
文件：
playbooks/ldap.yml
---
- hosts:
ldap
become:
yes
roles:
- ldap
执行：
ansible-playbook playbooks/ldap.yml
二、Keycloak SSO
部署：
Docker Compose。
目录：
mkdir -p docker-compose/keycloak
Keycloak Compose
文件：
docker-compose/keycloak/docker-compose.yml
services:
postgres:
image:
postgres:16
restart:
always
environment:
POSTGRES_DB:
keycloak
POSTGRES_USER:
keycloak
POSTGRES_PASSWORD:
password
volumes:
- /data/keycloak/postgres:/var/lib/postgresql/data
keycloak:
image:
quay.io/keycloak/keycloak:latest
restart:
always
command:
start-dev
environment:
KC_DB:
postgres
KC_DB_URL:
jdbc:postgresql://postgres:5432/keycloak
KC_DB_USERNAME:
keycloak
KC_DB_PASSWORD:
password
KEYCLOAK_ADMIN:
admin
KEYCLOAK_ADMIN_PASSWORD:
Admin@123456
ports:
- "8080:8080"
depends_on:
- postgres
Keycloak配置
登录：
http://server:8080
创建：
Realm:
company
LDAP Federation:
User Federation
Add provider
ldap
配置：
Connection URL:
ldap://openldap:389
Bind DN:
cn=admin,dc=company,dc=local
Users DN:
ou=users,dc=company,dc=local
Keycloak Role
目录：
mkdir -p roles/keycloak/{tasks,templates}
tasks/main.yml
---
- name:
deploy keycloak
template:
src:
docker-compose.yml.j2
dest:
/opt/keycloak/docker-compose.yml
- name:
start keycloak
community.docker.docker_compose_v2:
project_src:
/opt/keycloak
Keycloak Playbook
文件：
playbooks/keycloak.yml
---
- hosts:
keycloak
roles:
- keycloak
三、Vault
用途：
管理：
- 数据库密码
- LDAP密码
- TLS证书
- API Token
- Kubernetes Secret
架构：
Application
|
Vault Agent
|
Vault Server
Vault Compose
目录：
mkdir -p docker-compose/vault
文件：
docker-compose/vault/docker-compose.yml
services:
vault:
image:
hashicorp/vault:latest
container_name:
vault
restart:
always
cap_add:
- IPC_LOCK
ports:
- "8200:8200"
volumes:
- /data/vault:/vault/data
- ./vault.hcl:/vault/config/vault.hcl
command:
vault server -config=/vault/config/vault.hcl
Vault配置
文件：
vault.hcl
storage "file" {
path = "/vault/data"
}
listener "tcp" {
address = "0.0.0.0:8200"
tls_disable = true
}
ui = true
Vault初始化
启动：
docker compose up -d
初始化：
vault operator init
保存：
Unseal Key
Root Token
解封：
vault operator unseal
Vault LDAP认证
启用：
vault auth enable ldap
配置：
vault write auth/ldap/config \
url="ldap://openldap:389" \
userdn="ou=users,dc=company,dc=local" \
groupdn="ou=groups,dc=company,dc=local"
Vault Role
目录：
mkdir -p roles/vault/{tasks,templates}
tasks/main.yml
---
- name:
create vault dir
file:
path:
/opt/vault
state:
directory
- name:
deploy vault
template:
src:
docker-compose.yml.j2
dest:
/opt/vault/docker-compose.yml
- name:
start vault
community.docker.docker_compose_v2:
project_src:
/opt/vault
Vault Playbook
文件：
playbooks/vault.yml
---
- hosts:
vault
roles:
- vault
四、执行顺序
身份认证层部署：
# 1 LDAP
ansible-playbook playbooks/ldap.yml
# 2 Keycloak
ansible-playbook playbooks/keycloak.yml
# 3 Vault
ansible-playbook playbooks/vault.yml
Part 3：DevOps 层
本部分包含：
DevOps Platform
├── GitLab
│ ├── Git Repository
│ ├── CI/CD
│ ├── LDAP Login
│ ├── SMTP
│ └── Backup
├── Gerrit
│ ├── Code Review
│ ├── Git SSH
│ └── LDAP
├── Jenkins
│ ├── Pipeline
│ ├── Docker Agent
│ ├── LDAP
│ └── GitLab Integration
├── Nexus
│ ├── Maven Repository
│ ├── npm Repository
│ ├── Docker Registry
│ └── LDAP
├── SonarQube
│ ├── Code Scan
│ ├── Jenkins Integration
│ └── LDAP
└── Harbor
├── Docker Registry
├── LDAP
├── Scanner
└── Kubernetes Integration
一、目录增加
enterprise-platform-ansible/
├── playbooks/
│
├── gitlab.yml
├── gerrit.yml
├── jenkins.yml
├── nexus.yml
├── sonar.yml
└── harbor.yml
├── roles/
├── gitlab/
├── gerrit/
├── jenkins/
├── nexus/
├── sonar/
└── harbor/
└── docker-compose/
├── gitlab/
├── gerrit/
├── jenkins/
├── nexus/
├── sonar/
└── harbor/
二、GitLab
1. 创建目录
mkdir -p docker-compose/gitlab
GitLab Compose
文件：
docker-compose/gitlab/docker-compose.yml
services:
gitlab:
image: gitlab/gitlab-ee:latest
container_name: gitlab
restart: always
hostname:
gitlab.company.local
environment:
GITLAB_OMNIBUS_CONFIG: |
external_url 'https://gitlab.company.local'
gitlab_rails['ldap_enabled'] = true
gitlab_rails['ldap_servers'] = {
'main' => {
'label' => 'LDAP',
'host' => 'openldap',
'port' => 389,
'uid' => 'uid',
'bind_dn' => 'cn=admin,dc=company,dc=local',
'password' => 'Admin@123456',
'base' => 'ou=users,dc=company,dc=local'
}
}
gitlab_rails['smtp_enable']=true
ports:
- "80:80"
- "443:443"
- "2222:22"
volumes:
- /data/gitlab/config:/etc/gitlab
- /data/gitlab/logs:/var/log/gitlab
- /data/gitlab/data:/var/opt/gitlab
GitLab Role
目录：
mkdir -p roles/gitlab/tasks
roles/gitlab/tasks/main.yml
---
- name:
create gitlab directory
file:
path:
/opt/gitlab
state:
directory
- name:
copy compose
copy:
src:
docker-compose.yml
dest:
/opt/gitlab/docker-compose.yml
- name:
start gitlab
community.docker.docker_compose_v2:
project_src:
/opt/gitlab
GitLab Playbook
文件：
playbooks/gitlab.yml
---
- hosts:
gitlab
become:
yes
roles:
- gitlab
执行：
ansible-playbook playbooks/gitlab.yml
三、Gerrit
Compose
目录：
mkdir docker-compose/gerrit
文件：
docker-compose/gerrit/docker-compose.yml
services:
gerrit:
image:
gerritcodereview/gerrit:latest
container_name:
gerrit
restart:
always
ports:
- "8080:8080"
- "29418:29418"
volumes:
- /data/gerrit:/var/gerrit/review_site
Gerrit初始化
进入：
docker exec -it gerrit bash
安装LDAP插件：
git clone \
https://gerrit.googlesource.com/plugins/ldap
Gerrit配置
文件：
gerrit.config
[ldap]
server = ldap://openldap:389
username = cn=admin,dc=company,dc=local
password = Admin@123456
accountBase = ou=users,dc=company,dc=local
Gerrit Role
---
- name:
deploy gerrit
community.docker.docker_compose_v2:
project_src:
/opt/gerrit
Gerrit Playbook
---
- hosts:
gerrit
roles:
- gerrit
四、Jenkins
架构：
Jenkins
+
Docker Agent
+
GitLab
+
Sonar
+
Harbor
Compose
目录：
mkdir docker-compose/jenkins
文件：
docker-compose/jenkins/docker-compose.yml
services:
jenkins:
image:
jenkins/jenkins:lts
container_name:
jenkins
restart:
always
user:
root
ports:
- "8080:8080"
- "50000:50000"
volumes:
- /data/jenkins:/var/jenkins_home
- /var/run/docker.sock:/var/run/docker.sock
Jenkins插件
自动安装：
git
gitlab-plugin
pipeline
docker-plugin
sonar
kubernetes
Jenkins LDAP
配置：
Manage Jenkins
Security
LDAP
参数：
Server:
ldap://openldap:389
Root DN:
dc=company,dc=local
五、Nexus
Compose
services:
nexus:
image:
sonatype/nexus3
container_name:
nexus
restart:
always
ports:
- "8081:8081"
volumes:
- /data/nexus:/nexus-data
Nexus支持
仓库：
maven-releases
maven-snapshots
npm-private
docker-hosted
docker-proxy
六、SonarQube
Compose
services:
postgres:
image:
postgres:15
sonar:
image:
sonarqube:lts-community
ports:
- "9000:9000"
environment:
SONAR_JDBC_URL:
jdbc:postgresql://postgres:5432/sonar
Jenkins集成
Jenkinsfile:
pipeline {
stages {
stage('Sonar Scan'){
steps{
sh '''
sonar-scanner \
-Dsonar.projectKey=test
'''
}
}
}
}
七、Harbor
推荐：
官方离线安装包 + Ansible自动化。
目录：
roles/harbor
安装：
wget \
https://github.com/goharbor/harbor/releases/download/v2.x.x/harbor-offline-installer.tgz
Harbor配置
harbor.yml:
hostname:
harbor.company.local
https:
port:
443
database:
password:
Harbor@123456
data_volume:
/data/harbor
Harbor LDAP
配置：
Administration
Authentication
LDAP
参数：
LDAP URL:
ldap://openldap:389
Base DN:
ou=users,dc=company,dc=local
八、Playbook汇总
gitlab.yml
---
- hosts:
gitlab
roles:
- gitlab
gerrit.yml
---
- hosts:
gerrit
roles:
- gerrit
jenkins.yml
---
- hosts:
jenkins
roles:
- jenkins
nexus.yml
---
- hosts:
nexus
roles:
- nexus
sonar.yml
---
- hosts:
sonar
roles:
- sonar
harbor.yml
---
- hosts:
harbor
roles:
- harbor
九、部署顺序
建议：
# 1 LDAP
ansible-playbook playbooks/ldap.yml
# 2 GitLab
ansible-playbook playbooks/gitlab.yml
# 3 Gerrit
ansible-playbook playbooks/gerrit.yml
# 4 Jenkins
ansible-playbook playbooks/jenkins.yml
# 5 Nexus
ansible-playbook playbooks/nexus.yml
# 6 Sonar
ansible-playbook playbooks/sonar.yml
# 7 Harbor
ansible-playbook playbooks/harbor.yml
Part 4 Office协作平台
本部分部署：
Office Platform
├── Jira
│ ├── Jira Software
│ ├── PostgreSQL
│ ├── LDAP Authentication
│ └── Backup
├── Confluence
│ ├── Confluence Wiki
│ ├── PostgreSQL
│ ├── LDAP
│ └── Attachment Storage
├── Nextcloud
│ ├── PostgreSQL
│ ├── Redis
│ ├── LDAP
│ ├── Collabora
│ └── OnlyOffice
├── Mail Server
│ ├── Postfix
│ ├── Dovecot
│ ├── Roundcube
│ └── LDAP
├── FTP Server
│ └── vsftpd
└── OpenVPN
├── Server
├── LDAP Authentication
└── Certificate Management
一、目录结构增加
enterprise-platform-ansible/
├── playbooks/
│
├── jira.yml
├── confluence.yml
├── nextcloud.yml
├── collabora.yml
├── onlyoffice.yml
├── mail.yml
├── ftp.yml
└── openvpn.yml
roles/
├── jira/
├── confluence/
├── nextcloud/
├── collabora/
├── onlyoffice/
├── mail/
├── ftp/
└── openvpn/
docker-compose/
├── jira/
├── confluence/
├── nextcloud/
├── collabora/
├── onlyoffice/
├── mail/
├── ftp/
└── openvpn/
二、Jira 部署
架构：
LDAP
|
|
Jira ---- PostgreSQL
1. Jira Compose
创建：
mkdir -p docker-compose/jira
文件：
docker-compose/jira/docker-compose.yml
services:
postgres:
image:
postgres:15
container_name:
jira-postgres
restart:
always
environment:
POSTGRES_DB:
jira
POSTGRES_USER:
jira
POSTGRES_PASSWORD:
Jira@123456
volumes:
- /data/jira/postgres:/var/lib/postgresql/data
jira:
image:
atlassian/jira-software:latest
container_name:
jira
restart:
always
depends_on:
- postgres
ports:
- "8080:8080"
environment:
ATL_JDBC_URL:
jdbc:postgresql://postgres:5432/jira
ATL_JDBC_USER:
jira
ATL_JDBC_PASSWORD:
Jira@123456
volumes:
- /data/jira/data:/var/atlassian/application-data/jira
Jira LDAP配置
进入：
Jira
→
Administration
→
User Management
→
LDAP
配置：
LDAP URL:
ldap://openldap:389
Base DN:
dc=company,dc=local
User DN:
ou=users,dc=company,dc=local
Jira Role
目录：
mkdir -p roles/jira/tasks
roles/jira/tasks/main.yml
---
- name:
create jira dir
file:
path:
/opt/jira
state:
directory
- name:
deploy jira
community.docker.docker_compose_v2:
project_src:
/opt/jira
Jira Playbook
playbooks/jira.yml
---
- hosts:
jira
roles:
- jira
三、Confluence
架构：
LDAP
|
Confluence
|
PostgreSQL
|
NFS / MinIO Attachment Storage
Compose
目录：
mkdir -p docker-compose/confluence
文件：
docker-compose/confluence/docker-compose.yml
services:
postgres:
image:
postgres:15
environment:
POSTGRES_DB:
confluence
POSTGRES_USER:
confluence
POSTGRES_PASSWORD:
Confluence@123456
volumes:
- /data/confluence/postgres:/var/lib/postgresql/data
confluence:
image:
atlassian/confluence-server:latest
restart:
always
ports:
- "8090:8090"
environment:
ATL_JDBC_URL:
jdbc:postgresql://postgres:5432/confluence
ATL_JDBC_USER:
confluence
ATL_JDBC_PASSWORD:
Confluence@123456
volumes:
- /data/confluence/data:/var/atlassian/application-data/confluence
四、Nextcloud
架构：
LDAP
|
Client ---> Nextcloud
|
+--------+--------+
| |
PostgreSQL Redis
|
Collabora
|
OnlyOffice
Nextcloud Compose
目录：
mkdir -p docker-compose/nextcloud
文件：
docker-compose/nextcloud/docker-compose.yml
services:
postgres:
image:
postgres:15
environment:
POSTGRES_DB:
nextcloud
POSTGRES_USER:
nextcloud
POSTGRES_PASSWORD:
Nextcloud@123456
volumes:
- /data/nextcloud/postgres:/var/lib/postgresql/data
redis:
image:
redis:7
restart:
always
nextcloud:
image:
nextcloud:latest
restart:
always
ports:
- "8081:80"
depends_on:
- postgres
- redis
volumes:
- /data/nextcloud/html:/var/www/html
Nextcloud LDAP
配置：
Administration
LDAP/AD integration
参数：
Host:
openldap
Port:
389
Base DN:
dc=company,dc=local
五、Collabora
用于：
- 在线编辑 Word
- Excel
- PPT
Compose：
services:
collabora:
image:
collabora/code
restart:
always
ports:
- "9980:9980"
environment:
domain:
nextcloud.company.local
username:
admin
password:
password
六、OnlyOffice
Compose:
services:
onlyoffice:
image:
onlyoffice/documentserver
restart:
always
ports:
- "8082:80"
volumes:
- /data/onlyoffice:/var/www/onlyoffice/Data
Nextcloud集成
安装插件：
Apps
↓
Collabora Online
↓
OnlyOffice connector
七、邮件系统
采用：
- Postfix
- Dovecot
- Roundcube
架构：
User
|
Roundcube
|
Dovecot
|
Postfix
|
LDAP
Mail Compose
目录：
mkdir -p docker-compose/mail
文件：
docker-compose/mail/docker-compose.yml
services:
mailserver:
image:
mailserver/docker-mailserver
hostname:
mail.company.local
restart:
always
ports:
- "25:25"
- "143:143"
- "587:587"
- "993:993"
volumes:
- /data/mail/data:/var/mail
- /data/mail/config:/tmp/docker-mailserver/config
environment:
ENABLE_LDAP:
1
LDAP_SERVER_HOST:
openldap
八、FTP
采用：
vsftpd
Compose:
services:
ftp:
image:
fauria/vsftpd
restart:
always
ports:
- "21:21"
- "21100-21110:21100-21110"
environment:
FTP_USER:
ftpuser
FTP_PASS:
password
volumes:
- /data/ftp:/home/vsftpd
九、OpenVPN
架构：
Client
|
OpenVPN
|
LDAP
Compose
services:
openvpn:
image:
kylemanna/openvpn
container_name:
openvpn
restart:
always
ports:
- "1194:1194/udp"
volumes:
- /data/openvpn:/etc/openvpn
LDAP认证
安装插件：
openvpn-auth-ldap
配置：
<LDAP>
URL ldap://openldap:389
BindDN cn=admin,dc=company,dc=local
Password Admin@123456
BaseDN ou=users,dc=company,dc=local
</LDAP>
十、Playbook汇总
jira.yml
---
- hosts:
jira
roles:
- jira
confluence.yml
---
- hosts:
confluence
roles:
- confluence
nextcloud.yml
---
- hosts:
nextcloud
roles:
- nextcloud
mail.yml
---
- hosts:
mail
roles:
- mail
ftp.yml
---
- hosts:
ftp
roles:
- ftp
openvpn.yml
---
- hosts:
vpn
roles:
- openvpn
十一、部署顺序
建议：
# 1 PostgreSQL / LDAP基础
ansible-playbook playbooks/ldap.yml
# 2 Jira
ansible-playbook playbooks/jira.yml
# 3 Confluence
ansible-playbook playbooks/confluence.yml
# 4 Nextcloud
ansible-playbook playbooks/nextcloud.yml
# 5 Collabora
ansible-playbook playbooks/collabora.yml
# 6 OnlyOffice
ansible-playbook playbooks/onlyoffice.yml
# 7 Mail
ansible-playbook playbooks/mail.yml
# 8 FTP
ansible-playbook playbooks/ftp.yml
# 9 VPN
ansible-playbook playbooks/openvpn.yml
Part 5 Kubernetes层
本部分是整个企业平台的核心运行环境
目标：
- 自动化安装 Kubernetes 集群
- 支持 HA Master
- 支持 Keepalived + HAProxy
- 支持 containerd
- 支持 Helm
- 支持 Ingress
- 支持 ArgoCD GitOps
- 支持 Harbor 私有镜像仓库
- 支持 Velero / Restic 备份
Part 5 架构
VIP
10.50.0.100
|
+-------------+
| HAProxy |
+-------------+
| | |
| | |
+------+ +------+ +------+
| | | | | |
master1 master2 master3
Kubernetes API
|
+----------+-----------+
| |
Worker1 Worker2
Kubernetes Addons
|
+-- Nginx Ingress
|
+-- Cert-manager
|
+-- ArgoCD
|
+-- Harbor
|
+-- Velero
|
+-- Restic
一、目录增加
enterprise-platform-ansible/
├── playbooks/
│
├── kubernetes.yml
├── containerd.yml
├── kube-master.yml
├── kube-worker.yml
├── argocd.yml
├── ingress.yml
├── cert-manager.yml
├── velero.yml
└── restic.yml
roles/
├── containerd/
├── kubernetes/
├── kube-master/
├── kube-worker/
├── helm/
├── ingress-nginx/
├── cert-manager/
├── argocd/
├── velero/
└── restic/
二、Inventory扩展
文件：
inventory/production/hosts.ini
增加：
################################
# Kubernetes Cluster
################################
[k8s_master]
master01 ansible_host=10.50.0.40
master02 ansible_host=10.50.0.41
master03 ansible_host=10.50.0.42
[k8s_worker]
worker01 ansible_host=10.50.0.50
worker02 ansible_host=10.50.0.51
[k8s]
master01
master02
master03
worker01
worker02
三、Containerd安装
Kubernetes 1.31推荐：
containerd + runc + cni
创建：
mkdir -p roles/containerd/tasks
roles/containerd/tasks/main.yml
---
- name:
install containerd
apt:
name:
- containerd
- runc
state:
present
- name:
create config
shell:
cmd:
containerd config default > /etc/containerd/config.toml
- name:
enable systemd cgroup
replace:
path:
/etc/containerd/config.toml
regexp:
'SystemdCgroup = false'
replace:
'SystemdCgroup = true'
- name:
restart containerd
systemd:
name:
containerd
enabled:
yes
state:
restarted
Containerd Playbook
文件：
playbooks/containerd.yml
---
- hosts:
k8s
become:
yes
roles:
- containerd
执行：
ansible-playbook playbooks/containerd.yml
四、Kubernetes基础依赖
创建：
roles/kubernetes/tasks/main.yml
---
- name:
disable swap
command:
cmd:
swapoff -a
- name:
remove swap fstab
replace:
path:
/etc/fstab
regexp:
'^.*swap.*$'
replace:
''
- name:
install kube tools
apt:
name:
- kubelet
- kubeadm
- kubectl
state:
present
- name:
enable kubelet
systemd:
name:
kubelet
enabled:
yes
Kubernetes Role
roles/kubernetes/tasks/main.yml
包含：
kubeadm
kubelet
kubectl
sysctl
network
iptables
五、初始化Master
目录：
roles/kube-master/tasks
main.yml
---
- name:
initialize kubernetes
command:
cmd:
kubeadm init \
--control-plane-endpoint "{{ vip }}:6443" \
--upload-certs
args:
creates:
/etc/kubernetes/admin.conf
kubeadm配置
模板：
roles/kube-master/templates/kubeadm.yaml.j2
apiVersion:
kubeadm.k8s.io/v1beta4
kind:
ClusterConfiguration
kubernetesVersion:
v1.31.0
controlPlaneEndpoint:
"{{ vip }}:6443"
networking:
podSubnet:
10.244.0.0/16
六、Worker加入
roles/kube-worker/tasks/main.yml
---
- name:
join cluster
command:
cmd:
kubeadm join {{ vip }}:6443 \
--token {{ kube_token }} \
--discovery-token-ca-cert-hash {{ discovery_hash }}
七、Kubernetes安装总入口
文件：
playbooks/kubernetes.yml
---
- hosts:
k8s
become:
yes
roles:
- containerd
- kubernetes
- hosts:
k8s_master
roles:
- kube-master
- hosts:
k8s_worker
roles:
- kube-worker
八、Helm安装
目录：
roles/helm/tasks/main.yml
---
- name:
install helm
shell:
cmd:
curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
九、Ingress-Nginx
生产入口：
Internet
|
HAProxy
|
Ingress Controller
|
Service
安装：
helm repo add ingress-nginx \
https://kubernetes.github.io/ingress-nginx
helm install ingress-nginx \
ingress-nginx/ingress-nginx \
-n ingress-nginx \
--create-namespace
Ansible:
roles/ingress-nginx/tasks/main.yml
---
- name:
install ingress
command:
cmd:
helm install ingress-nginx \
ingress-nginx/ingress-nginx \
-n ingress-nginx \
--create-namespace
十、Cert-manager
用于：
- Let's Encrypt
- 内部CA
- TLS自动签发
安装：
helm repo add jetstack \
https://charts.jetstack.io
helm install cert-manager \
jetstack/cert-manager \
-n cert-manager \
--create-namespace \
--set installCRDs=true
十一、ArgoCD
GitOps核心。
架构：
GitLab
|
ArgoCD
|
Kubernetes
安装：
kubectl create namespace argocd
helm repo add argo \
https://argoproj.github.io/argo-helm
helm install argocd \
argo/argo-cd \
-n argocd
Playbook：
playbooks/argocd.yml
---
- hosts:
k8s_master
roles:
- argocd
十二、Harbor Kubernetes集成
使用：
imagePullSecrets
创建：
kubectl create secret docker-registry harbor-secret \
--docker-server=harbor.company.local \
--docker-username=admin \
--docker-password=password
应用：
imagePullSecrets:
- name:
harbor-secret
十三、Velero备份
支持：
- Kubernetes资源
- PV
- S3
- MinIO
安装：
velero install \
--provider aws \
--bucket k8s-backup \
--backup-location-config \
region=minio,s3ForcePathStyle=true,s3Url=http://minio:9000
备份：
velero backup create all-backup
恢复：
velero restore create \
--from-backup all-backup
十四、Restic
文件备份
安装：
apt install restic
初始化：
restic init \
-r /backup
备份：
restic backup /data
恢复：
restic restore latest
十五、Kubernetes完整部署顺序
执行：
1. 基础组件
ansible-playbook \
playbooks/containerd.yml
2. Kubernetes
ansible-playbook \
playbooks/kubernetes.yml
3. Helm
ansible-playbook \
playbooks/helm.yml
4. Ingress
ansible-playbook \
playbooks/ingress.yml
5. Cert Manager
ansible-playbook \
playbooks/cert-manager.yml
6. ArgoCD
ansible-playbook \
playbooks/argocd.yml
7. Backup
ansible-playbook \
playbooks/velero.yml
ansible-playbook \
playbooks/restic.yml
Part 6：Monitoring监控平台
本部分建设企业级可观测平台：
Monitoring Platform
Grafana
|
+-----------+------------+
| |
Prometheus Loki
| |
| Promtail
|
+-----+-------+
| |
Node Exporter App Metrics
|
|
Alertmanager
|
Email / Webhook / 企业微信
|
Thanos
|
Object Storage
(MinIO/S3)
ELK Stack:
Filebeat
|
Logstash
|
Elasticsearch
|
Kibana
一、目录增加
enterprise-platform-ansible/
├── playbooks/
│
├── prometheus.yml
├── alertmanager.yml
├── grafana.yml
├── loki.yml
├── elk.yml
└── thanos.yml
roles/
├── prometheus/
├── alertmanager/
├── grafana/
├── loki/
├── promtail/
├── elk/
└── thanos/
docker-compose/
├── prometheus/
├── grafana/
├── loki/
├── elk/
└── thanos/
二、Prometheus
推荐：
生产环境：
Prometheus Operator
+
kube-prometheus-stack
监控：
- Kubernetes
- Node
- Container
- Application
1. Helm安装
添加仓库：
helm repo add prometheus-community \
https://prometheus-community.github.io/helm-charts
helm repo update
安装：
helm install monitoring \
prometheus-community/kube-prometheus-stack \
-n monitoring \
--create-namespace
组件：
kube-prometheus-stack
├── Prometheus
├── Alertmanager
├── Grafana
├── Node Exporter
├── kube-state-metrics
└── Operator
三、Prometheus Ansible Role
目录：
mkdir -p roles/prometheus/tasks
roles/prometheus/tasks/main.yml
---
- name:
create namespace
command:
kubectl create namespace monitoring
ignore_errors:
yes
- name:
add repo
command:
helm repo add prometheus-community \
https://prometheus-community.github.io/helm-charts
- name:
install prometheus
command:
helm install monitoring \
prometheus-community/kube-prometheus-stack \
-n monitoring
Prometheus Playbook
文件：
playbooks/prometheus.yml
---
- hosts:
k8s_master
roles:
- prometheus
四、Alertmanager
功能：
Alert Rule
|
Alertmanager
|
+-----------+
|
Email
|
Webhook
|
DingTalk
|
WeChat
Alertmanager配置
文件：
alertmanager.yml
global:
smtp_smarthost:
mail.company.com:587
smtp_from:
alert@company.com
route:
receiver:
email
receivers:
- name:
email
email_configs:
- to:
admin@company.com
Ansible配置
roles:
roles/alertmanager/templates/
模板：
alertmanager.yml.j2
tasks:
---
- name:
apply alertmanager config
template:
src:
alertmanager.yml.j2
dest:
/tmp/alertmanager.yml
Playbook
playbooks/alertmanager.yml
---
- hosts:
k8s_master
roles:
- alertmanager
五、Grafana
功能：
- Dashboard
- LDAP登录
- Prometheus datasource
- Loki datasource
Helm安装
helm install grafana \
grafana/grafana \
-n monitoring
Grafana LDAP
配置：
文件：
grafana.ini
[auth.ldap]
enabled = true
config_file = /etc/grafana/ldap.toml
ldap.toml
[[servers]]
host = "openldap"
port = 389
bind_dn = "cn=admin,dc=company,dc=local"
search_base_dns = [
"ou=users,dc=company,dc=local"
]
Grafana Role
---
- name:
deploy grafana
command:
helm install grafana \
grafana/grafana \
-n monitoring
六、Loki日志平台
架构：
Container
|
Promtail
|
Loki
|
Grafana
Helm安装
添加：
helm repo add grafana \
https://grafana.github.io/helm-charts
安装：
helm install loki \
grafana/loki-stack \
-n monitoring
Loki配置
values.yaml:
loki:
persistence:
enabled:
true
size:
200Gi
promtail:
enabled:
true
Promtail采集
采集：
/var/log/*
/var/lib/docker/containers/*
七、ELK
架构：
Application
|
Filebeat
|
Logstash
|
Elasticsearch
|
Kibana
Docker Compose
目录：
mkdir -p docker-compose/elk
docker-compose.yml
services:
elasticsearch:
image:
elasticsearch:8.15.0
environment:
discovery.type:
single-node
xpack.security.enabled:
false
volumes:
- /data/elasticsearch:/usr/share/elasticsearch/data
ports:
- "9200:9200"
logstash:
image:
logstash:8.15.0
ports:
- "5044:5044"
kibana:
image:
kibana:8.15.0
ports:
- "5601:5601"
Filebeat配置
filebeat.yml
filebeat.inputs:
- type:
container
paths:
- /var/lib/docker/containers/*/*.log
output.logstash:
hosts:
- logstash:5044
ELK Role
目录：
roles/elk
tasks:
---
- name:
deploy elk
community.docker.docker_compose_v2:
project_src:
/opt/elk
ELK Playbook
---
- hosts:
elk
roles:
- elk
八、Thanos
作用：
解决：
Prometheus:
单节点
数据短期
Thanos:
多Prometheus
+
长期存储
+
全局查询
架构：
Prometheus-1
|
Thanos Sidecar
|
Object Storage
(MinIO)
|
Thanos Query
|
Grafana
Thanos组件
thanos-sidecar
thanos-query
thanos-store
thanos-compactor
Helm安装
helm repo add bitnami \
https://charts.bitnami.com/bitnami
helm install thanos \
bitnami/thanos \
-n monitoring
Thanos对象存储
配置：
objstore.yml
type:
s3
config:
bucket:
thanos
endpoint:
minio.company.local:9000
access_key:
admin
secret_key:
password
九、统一监控拓扑
最终：
Grafana
|
+---------------+---------------+
| |
Prometheus Loki
| |
| Promtail
|
Alertmanager
|
|
Thanos
|
MinIO
ELK:
Filebeat
|
Logstash
|
Elasticsearch
|
Kibana
十、部署顺序
1. Prometheus
ansible-playbook \
playbooks/prometheus.yml
2. Alertmanager
ansible-playbook \
playbooks/alertmanager.yml
3. Grafana
ansible-playbook \
playbooks/grafana.yml
4. Loki
ansible-playbook \
playbooks/loki.yml
5. ELK
ansible-playbook \
playbooks/elk.yml
6. Thanos
ansible-playbook \
playbooks/thanos.yml
十一、监控覆盖范围
部署完成后：
| 对象 | 组件 |
|---|---|
| 服务器CPU | Node Exporter |
| 服务器内存 | Node Exporter |
| 磁盘 | Node Exporter |
| Docker | cAdvisor |
| Kubernetes | kube-state-metrics |
| Pod | Prometheus |
| 应用 | Exporter |
| 日志 | Loki |
| 审计日志 | ELK |
| 长期指标 | Thanos |
| 告警 | Alertmanager |
Part 7：Backup备份层
本部分建设企业级备份体系：
目标：
- Kubernetes 集群备份
- PVC 数据备份
- MySQL/PostgreSQL/MongoDB备份
- MinIO/S3对象存储备份
- 文件系统备份
- 加密备份
- 定时备份
- 灾难恢复
Part 7 架构
Backup Platform
|
+---------------+----------------+
| |
Velero Restic
| |
| |
Kubernetes Resources File System Backup
|
|
+-----+------+
| |
PV PVC
|
|
Object Storage
MinIO / S3
|
|
Long Term Storage
一、目录增加
enterprise-platform-ansible/
├── playbooks/
│
├── velero.yml
├── restic.yml
├── database-backup.yml
├── minio-backup.yml
└── restore.yml
roles/
├── velero/
├── restic/
├── database-backup/
├── minio-backup/
└── restore/
二、备份规划
企业环境建议：
| 系统 | 备份方式 |
|---|---|
| Kubernetes资源 | Velero |
| PVC数据 | Velero + Restic |
| MySQL | mysqldump |
| PostgreSQL | pg_dump |
| MongoDB | mongodump |
| Redis | RDB |
| MinIO | mc mirror |
| 文件服务器 | Restic |
| GitLab | 官方backup |
| Jira | 数据库+附件 |
| Confluence | 数据库+附件 |
三、Velero Kubernetes备份
1. 安装客户端
wget \
https://github.com/vmware-tanzu/velero/releases/download/v1.15.0/velero-v1.15.0-linux-amd64.tar.gz
tar xf velero*.tar.gz
cp velero*/velero /usr/local/bin/
四、MinIO作为备份仓库
架构：
Kubernetes
|
Velero
|
S3 API
|
MinIO
|
/backup/velero
创建bucket：
mc alias set minio \
http://minio.company.local:9000 \
admin password
mc mb minio/k8s-backup
五、Velero安装
安装：
velero install \
--provider aws \
--plugins velero/velero-plugin-for-aws:v1.10.0 \
--bucket k8s-backup \
--secret-file ./credentials \
--backup-location-config \
region=minio,\
s3ForcePathStyle=true,\
s3Url=http://minio.company.local:9000 \
--use-node-agent
credentials
文件：
credentials
[default]
aws_access_key_id=admin
aws_secret_access_key=password
六、Velero Ansible Role
目录：
roles/velero/tasks
main.yml
---
- name:
create backup namespace
command:
kubectl create namespace velero
ignore_errors:
yes
- name:
install velero
shell:
cmd: |
velero install \
--provider aws \
--bucket k8s-backup \
--use-node-agent \
--secret-file /tmp/credentials \
--namespace velero
Velero Playbook
---
- hosts:
k8s_master
become:
yes
roles:
- velero
七、创建备份任务
全量备份
velero backup create cluster-backup
查看：
velero backup get
结果：
NAME
cluster-backup
STATUS
Completed
八、定时备份
每天凌晨2点：
velero schedule create daily-backup \
--schedule="0 2 * * *"
查看：
velero schedule get
九、恢复
模拟灾难：
删除namespace：
kubectl delete ns production
恢复：
velero restore create \
--from-backup cluster-backup
查看：
velero restore get
十、Restic文件备份
适合：
/data
/home
/upload
附件
日志
配置文件
安装
Ubuntu:
apt install restic
CentOS:
yum install restic
Restic仓库
创建：
mkdir /backup/restic
初始化：
restic \
-r /backup/restic \
init
设置密码：
export RESTIC_PASSWORD=password
十一、Restic备份脚本
目录：
roles/restic/files/
文件：
backup.sh
内容：
#!/bin/bash
export RESTIC_PASSWORD=password
RESTIC_REPOSITORY=/backup/restic
restic backup \
/data \
--exclude=/data/tmp
restic forget \
--keep-daily 7 \
--keep-weekly 4 \
--keep-monthly 12 \
--prune
授权：
chmod +x backup.sh
十二、Systemd定时
文件：
/etc/systemd/system/restic-backup.service
[Unit]
Description=Restic Backup
[Service]
Type=oneshot
ExecStart=/usr/local/bin/restic-backup.sh
Timer:
restic-backup.timer
[Timer]
OnCalendar=*-*-* 03:00:00
Persistent=true
[Install]
WantedBy=timers.target
启动：
systemctl enable --now restic-backup.timer
十三、数据库备份
PostgreSQL
脚本：
#!/bin/bash
pg_dumpall \
-h postgres \
-U postgres \
> /backup/postgres.sql
MySQL
mysqldump \
-u root \
-p \
--all-databases \
> mysql.sql
MongoDB
mongodump \
--out /backup/mongo
十四、数据库Ansible
目录：
roles/database-backup
tasks:
---
- name:
install backup tools
apt:
name:
- postgresql-client
- mysql-client
state:
present
- name:
deploy backup script
copy:
src:
backup.sh
dest:
/usr/local/bin/db-backup.sh
mode:
0755
十五、MinIO备份
使用：
mc mirror
脚本：
#!/bin/bash
mc alias set source \
http://minio01:9000 \
admin password
mc alias set backup \
http://minio02:9000 \
admin password
mc mirror \
--overwrite \
source/allin \
backup/allin
十六、GitLab备份
GitLab自带：
gitlab-backup create
产生：
/var/opt/gitlab/backups
备份：
restic backup \
/var/opt/gitlab/backups
十七、Jira / Confluence备份
数据：
PostgreSQL
+
Attachment目录
备份：
restic backup \
/data/jira
/data/confluence
十八、灾难恢复流程
完整恢复顺序：
1. 恢复基础服务器
|
2. 恢复Kubernetes
|
3. 安装ArgoCD
|
4. Velero恢复资源
|
5. Restic恢复数据
|
6. 恢复数据库
|
7. 恢复应用
十九、Backup Playbook汇总
velero.yml
---
- hosts:
k8s_master
roles:
- velero
restic.yml
---
- hosts:
storage
roles:
- restic
database-backup.yml
---
- hosts:
database
roles:
- database-backup
二十、企业备份策略
推荐：
日备份
每天02:00
Kubernetes
数据库
文件
周备份
每周日
完整快照
月归档
保留12个月
Part 8：安全与身份认证层
本部分建设企业级安全体系：
目标：
- 统一身份认证（SSO）
- LDAP 用户中心
- OAuth2/OIDC
- Secret 密钥管理
- Kubernetes Secret 自动注入
- VPN 身份认证
- TLS 自动化
- 安全审计
- 主机入侵检测
Part 8 架构
Security Platform
|
+---------------------+---------------------+
| | |
OpenLDAP Keycloak Vault
| | |
| | |
User Center SSO/OIDC Secret Store
| | |
| | |
+----------+----------+---------------------+
|
Applications
GitLab
Jenkins
Grafana
Harbor
Jira
Confluence
Nextcloud
Kubernetes
|
Cert-manager
|
TLS
|
Wazuh
|
Security Audit
一、目录增加
enterprise-platform-ansible/
├── playbooks/
│
├── ldap.yml
├── keycloak.yml
├── vault.yml
├── cert-manager.yml
├── wazuh.yml
├── security-scan.yml
└── mfa.yml
roles/
├── openldap/
├── keycloak/
├── vault/
├── cert-manager/
├── wazuh/
├── trivy/
└── falco/
二、OpenLDAP统一身份中心
架构
LDAP
|
+--------------+--------------+
| | |
GitLab Jenkins Harbor
|
Grafana
|
Jira
|
Confluence
三、OpenLDAP Docker Compose
目录：
mkdir -p docker-compose/openldap
文件：
docker-compose/openldap/docker-compose.yml
services:
openldap:
image:
osixia/openldap:latest
container_name:
openldap
restart:
always
environment:
LDAP_ORGANISATION:
Company
LDAP_DOMAIN:
company.local
LDAP_ADMIN_PASSWORD:
Admin@123456
ports:
- "389:389"
- "636:636"
volumes:
- /data/openldap/data:/var/lib/ldap
- /data/openldap/config:/etc/ldap/slapd.d
LDAP目录设计
推荐：
dc=company,dc=local
├── ou=users
│
├── user001
├── user002
├── ou=groups
│
├── devops
├── developers
└── administrators
LDAP初始化LDIF
文件：
users.ldif
dn: ou=users,dc=company,dc=local
objectClass: organizationalUnit
ou: users
dn: ou=groups,dc=company,dc=local
objectClass: organizationalUnit
ou: groups
导入：
ldapadd \
-x \
-D "cn=admin,dc=company,dc=local" \
-W \
-f users.ldif
四、Keycloak SSO
作用：
一次登录
↓
所有系统认证
GitLab
Jenkins
Grafana
Harbor
Kubernetes
Keycloak架构
LDAP
|
Keycloak
|
+---------+---------+
| | |
GitLab Grafana Jenkins
Keycloak Compose
目录：
mkdir docker-compose/keycloak
docker-compose.yml
services:
postgres:
image:
postgres:15
environment:
POSTGRES_DB:
keycloak
POSTGRES_USER:
keycloak
POSTGRES_PASSWORD:
Keycloak@123456
keycloak:
image:
quay.io/keycloak/keycloak:latest
command:
start-dev
ports:
- "8080:8080"
environment:
KC_DB:
postgres
KC_DB_URL:
jdbc:postgresql://postgres:5432/keycloak
KC_DB_USERNAME:
keycloak
KC_DB_PASSWORD:
Keycloak@123456
KEYCLOAK_ADMIN:
admin
KEYCLOAK_ADMIN_PASSWORD:
admin123
Keycloak LDAP Federation
配置：
Realm
↓
User Federation
↓
LDAP
参数：
Connection URL:
ldap://openldap:389
Bind DN:
cn=admin,dc=company,dc=local
Users DN:
ou=users,dc=company,dc=local
五、Vault 密钥管理
作用：
替代：
明文密码
yaml secret
.env文件
管理：
数据库密码
API Token
SSH Key
TLS证书
Vault架构
Application
|
Vault Agent
|
Vault Server
|
Encrypted Storage
Vault Compose
services:
vault:
image:
hashicorp/vault:latest
container_name:
vault
ports:
- "8200:8200"
cap_add:
- IPC_LOCK
environment:
VAULT_DEV_ROOT_TOKEN_ID:
root
command:
server -dev
Vault初始化
进入：
vault login root
开启KV：
vault secrets enable kv
保存密码：
vault kv put secret/database \
password=Pass123
读取：
vault kv get secret/database
六、Kubernetes Vault Integration
安装：
helm repo add hashicorp \
https://helm.releases.hashicorp.com
helm install vault \
hashicorp/vault \
-n vault \
--create-namespace
Secret自动注入
Pod:
annotations:
vault.hashicorp.com/agent-inject:
"true"
vault.hashicorp.com/role:
app
应用启动自动获得：
/vault/secrets/database
七、Cert-manager TLS
用于：
HTTPS
Ingress TLS
内部CA
自动续期
安装：
helm repo add jetstack \
https://charts.jetstack.io
helm install cert-manager \
jetstack/cert-manager \
-n cert-manager \
--create-namespace \
--set installCRDs=true
ClusterIssuer
文件：
letsencrypt.yaml
apiVersion:
cert-manager.io/v1
kind:
ClusterIssuer
metadata:
name:
letsencrypt
spec:
acme:
email:
admin@company.com
server:
https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name:
letsencrypt-key
solvers:
- http01:
ingress:
八、Wazuh安全审计
功能：
服务器监控
文件完整性
漏洞检测
日志分析
入侵检测
Wazuh架构
Agent
|
Manager
|
Indexer
|
Dashboard
Docker Compose
services:
wazuh-manager:
image:
wazuh/wazuh-manager
ports:
- "1514:1514"
- "55000:55000"
wazuh-indexer:
image:
wazuh/wazuh-indexer
wazuh-dashboard:
image:
wazuh/wazuh-dashboard
ports:
- "5601:5601"
九、Trivy漏洞扫描
扫描：
Docker Image
Kubernetes
Filesystem
安装：
apt install trivy
扫描镜像：
trivy image nginx:latest
Harbor集成
开启：
Harbor
↓
Scanner
↓
Trivy
扫描：
Critical
High
Medium
Low
十、Falco运行时安全
Kubernetes Runtime Security
安装：
helm repo add falcosecurity \
https://falcosecurity.github.io/charts
helm install falco \
falcosecurity/falco \
-n falco
检测：
Shell进入容器
异常网络
异常文件修改
权限提升
十一、MFA多因素认证
推荐：
Keycloak
+
Google Authenticator
+
Duo
流程：
User Login
↓
Password
↓
OTP
↓
Access
十二、安全Playbook
ldap.yml
---
- hosts:
ldap
roles:
- openldap
keycloak.yml
---
- hosts:
keycloak
roles:
- keycloak
vault.yml
---
- hosts:
vault
roles:
- vault
wazuh.yml
---
- hosts:
security
roles:
- wazuh
十三、安全部署顺序
# 1 LDAP
ansible-playbook ldap.yml
# 2 Keycloak
ansible-playbook keycloak.yml
# 3 Vault
ansible-playbook vault.yml
# 4 Cert-manager
ansible-playbook cert-manager.yml
# 5 Wazuh
ansible-playbook wazuh.yml
# 6 Trivy
ansible-playbook security-scan.yml
十四、最终认证拓扑
完成后：
Keycloak
|
+-----------+------------+
| | |
GitLab Jenkins Grafana
|
Harbor
|
Kubernetes
LDAP
|
User Database
Vault
|
Secrets
Cert-manager
|
TLS
Wazuh
|
Security Audit
Part 8：安全与身份认证层
Part 8 Security 安全层（Enterprise Production Edition）
目录扩展
enterprise-platform-ansible/
roles/
├── openldap-ha
├── keycloak-ha
├── vault-ha
├── kubernetes-security
├── rbac
├── network-policy
├── pod-security
├── image-security
├── os-hardening
├── audit
└── compliance
playbooks/
├── ldap-ha.yml
├── keycloak-ha.yml
├── vault-ha.yml
├── k8s-security.yml
├── os-hardening.yml
└── compliance.yml
一、LDAP 高可用
生产环境：
Keepalived VIP
|
+----------------+
| LDAP VIP |
+----------------+
| |
LDAP01 LDAP02
| |
Replication
OpenLDAP HA Compose
services:
ldap01:
image: osixia/openldap
hostname:
ldap01.company.local
environment:
LDAP_DOMAIN:
company.local
LDAP_ADMIN_PASSWORD:
password
ldap02:
image:
osixia/openldap
hostname:
ldap02.company.local
environment:
LDAP_DOMAIN:
company.local
LDAP_ADMIN_PASSWORD:
password
LDAP同步
启用：
syncrepl
ldap01:
olcSyncrepl:
rid=001
provider=ldap://ldap02.company.local
searchbase="dc=company,dc=local"
binddn="cn=admin,dc=company,dc=local"
ldap02:
olcSyncrepl:
rid=002
provider=ldap://ldap01.company.local
二、Keycloak生产HA
架构：
HAProxy
|
+---------+---------+
| |
Keycloak01 Keycloak02
| |
+---------+---------+
|
PostgreSQL HA
Keycloak Compose
services:
keycloak:
image:
quay.io/keycloak/keycloak
replicas:
2
command:
start
environment:
KC_PROXY:
edge
KC_DB:
postgres
KC_DB_URL:
jdbc:postgresql://postgres:5432/keycloak
Keycloak安全配置
开启：
Realm Settings
↓
Security Defenses
Brute Force Detection
ON
Password Policy
ON
OTP Required
ON
三、Vault HA
生产禁止：
-dev模式
采用：
Vault Raft Cluster
架构：
HAProxy
|
+----------+----------+
| | |
Vault01 Vault02 Vault03
Raft
Vault配置
文件：
vault.hcl
storage "raft" {
path="/vault/data"
}
listener "tcp" {
address="0.0.0.0:8200"
tls_disable=false
}
cluster_addr="https://vault01:8201"
api_addr="https://vault01:8200"
ui=true
Vault初始化
第一台：
vault operator init
生成：
Unseal Key
Root Token
加入节点：
vault operator raft join \
https://vault01:8200
查看集群：
vault operator raft list-peers
输出：
vault01 leader
vault02 follower
vault03 follower
四、Kubernetes安全加固
1. RBAC
目录：
roles/rbac
创建管理员：
apiVersion:
rbac.authorization.k8s.io/v1
kind:
ClusterRoleBinding
metadata:
name:
platform-admin
subjects:
- kind:
User
name:
admin
roleRef:
kind:
ClusterRole
name:
cluster-admin
apiGroup:
rbac.authorization.k8s.io
五、ServiceAccount安全
禁止默认账户：
automountServiceAccountToken:
false
示例：
apiVersion:
v1
kind:
ServiceAccount
metadata:
name:
app
automountServiceAccountToken:
false
六、NetworkPolicy
实现：
Pod隔离
东西向流量控制
微服务安全
默认拒绝：
apiVersion:
networking.k8s.io/v1
kind:
NetworkPolicy
metadata:
name:
deny-all
spec:
podSelector:{}
policyTypes:
- Ingress
- Egress
允许web访问api：
kind:
NetworkPolicy
spec:
podSelector:
matchLabels:
app:
api
ingress:
- from:
- podSelector:
matchLabels:
app:
web
七、Pod Security Standard
启用：
Namespace:
kubectl label ns production \
pod-security.kubernetes.io/enforce=restricted
级别：
privileged
baseline
restricted
生产推荐：
restricted
八、镜像安全
Harbor + Trivy
流程：
Developer
|
GitLab CI
|
Build Image
|
Harbor
|
Trivy Scan
|
PASS
|
Deploy Kubernetes
GitLab CI示例：
security_scan:
stage:
test
script:
- trivy image \
$IMAGE
allow_failure:
false
九、Kubernetes Secret加密
默认：
Secret Base64
生产：
EncryptionConfiguration
文件：
encryption.yaml
apiVersion:
apiserver.config.k8s.io/v1
kind:
EncryptionConfiguration
resources:
- resources:
- secrets
providers:
- aescbc:
keys:
- name:
key1
secret:
xxx
- identity:{}
修改：
kube-apiserver
增加：
--encryption-provider-config=/etc/kubernetes/encryption.yaml
十、SSH安全加固
Ansible:
roles/os-hardening
禁用密码登录：
- name:
disable ssh password
lineinfile:
path:
/etc/ssh/sshd_config
regexp:
PasswordAuthentication
line:
PasswordAuthentication no
禁止root登录：
PermitRootLogin no
十一、Linux安全基线
安装：
apt install auditd
apt install fail2ban
apt install apparmor
十二、Auditd
监控：
/etc/passwd
/etc/shadow
/etc/sudoers
ssh配置
规则：
/etc/audit/rules.d/security.rules
-w /etc/passwd -p wa
-w /etc/shadow -p wa
-w /etc/sudoers -p wa
十三、Fail2ban
保护：
SSH
Nginx
VPN
配置：
jail.local
[sshd]
enabled=true
maxretry=5
bantime=3600
十四、漏洞扫描
Trivy
扫描：
镜像
trivy image nginx:latest
文件系统
trivy fs /
Kubernetes
trivy k8s cluster
十五、CIS Kubernetes Benchmark
工具：
kube-bench
安装：
curl -L \
https://github.com/aquasecurity/kube-bench/releases \
运行：
kube-bench run
检查：
PASS
WARN
FAIL
十六、Falco运行时安全
检测：
容器逃逸
异常Shell
敏感文件读取
权限提升
规则：
- rule:
Terminal shell in container
condition:
container.id != host
output:
Shell executed
十七、安全监控接入
统一进入：
Grafana
+
Loki
+
Prometheus
+
Wazuh
监控：
| 项目 | 组件 |
|---|---|
| 登录异常 | Wazuh |
| 漏洞 | Trivy |
| 容器异常 | Falco |
| 系统日志 | Loki |
| 指标 | Prometheus |
| 认证 | Keycloak |
| 密码 | Vault |
十八、安全部署总流程
# LDAP
ansible-playbook ldap-ha.yml
# SSO
ansible-playbook keycloak-ha.yml
# Secret
ansible-playbook vault-ha.yml
# Kubernetes安全
ansible-playbook k8s-security.yml
# OS加固
ansible-playbook os-hardening.yml
# 合规扫描
ansible-playbook compliance.yml
Part 9：Network & High Availability 网络高可用层
本部分建设企业生产级网络基础设施：
目标：
- 企业 DNS 服务
- 内外网代理
- 四层/七层负载均衡
- VIP 高可用
- VPN 接入
- 防火墙策略
- FTP/SFTP 文件交换
- Kubernetes Ingress 高可用
- 网络自动化部署
一、Network & HA 总体架构
Internet
|
Firewall
|
+---------------+
| HAProxy VIP |
+---------------+
/ \
/ \
HAProxy01 HAProxy02
| |
+----------------+
|
Kubernetes
|
+-------------+-------------+
| |
Ingress Services
|
Internal Network
DNS
|
Bind9
Proxy
|
Squid/Nginx
VPN
|
OpenVPN
File Transfer
|
SFTP/FTP
二、目录增加
enterprise-platform-ansible/
roles/
├── bind9/
├── dnsmasq/
├── nginx-proxy/
├── squid/
├── haproxy/
├── keepalived/
├── openvpn/
├── wireguard/
├── vsftpd/
├── sftp/
├── firewall/
└── network-check/
playbooks/
├── dns.yml
├── proxy.yml
├── haproxy.yml
├── keepalived.yml
├── vpn.yml
├── ftp.yml
└── firewall.yml
三、DNS 服务
企业推荐：
主DNS:
Bind9
备用DNS:
Bind9
内部DNS:
CoreDNS
DNS 架构
Client
|
DNS VIP
|
+---------+---------+
| |
DNS01 DNS02
Zone Replication
四、Bind9 Docker Compose
目录：
docker-compose/bind9
docker-compose.yml
services:
bind9:
image:
internetsystemsconsortium/bind9:latest
container_name:
bind9
restart:
always
ports:
- "53:53/tcp"
- "53:53/udp"
volumes:
- /data/bind9/config:/etc/bind
- /data/bind9/cache:/var/cache/bind
五、DNS Zone
文件：
company.local.zone
$TTL 86400
@ IN SOA dns01.company.local.
admin.company.local.
(
2026090901
3600
1800
604800
86400
)
@ IN NS dns01.company.local.
@ IN NS dns02.company.local.
dns01 IN A 10.50.0.10
dns02 IN A 10.50.0.11
gitlab IN A 10.50.0.20
jenkins IN A 10.50.0.21
harbor IN A 10.50.0.22
grafana IN A 10.50.0.23
六、DNS Ansible Role
目录：
roles/bind9/tasks/main.yml
---
- name:
install bind9
package:
name:
- bind9
state:
present
- name:
start bind9
service:
name:
bind9
state:
started
enabled:
yes
七、Proxy代理层
包含：
正向代理
Squid
反向代理
Nginx
八、Squid Proxy
用途：
- 软件包下载
- Docker镜像代理
- 内网出口控制
架构：
Client
|
Squid
|
Internet
Docker Compose
services:
squid:
image:
sameersbn/squid
ports:
- "3128:3128"
volumes:
- /data/squid:/etc/squid
Squid ACL
配置：
acl company src 10.50.0.0/16
http_access allow company
http_access deny all
九、Nginx Reverse Proxy
用于：
https://gitlab.company.com
|
Nginx
|
GitLab Container
配置：
server {
listen 443 ssl;
server_name gitlab.company.com;
location / {
proxy_pass http://gitlab:80;
proxy_set_header Host $host;
}
}
十、HAProxy
用途：
- Kubernetes API
- TCP Load Balance
- HTTP Load Balance
架构：
VIP
|
HAProxy Cluster
/ \
node01 node02
HAProxy Compose
services:
haproxy:
image:
haproxy:3
ports:
- "80:80"
- "443:443"
- "6443:6443"
volumes:
- ./haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg
HAProxy配置
文件：
haproxy.cfg
Kubernetes API
frontend kubernetes-api
bind *:6443
mode tcp
default_backend k8s-master
backend k8s-master
mode tcp
balance roundrobin
server master01 10.50.0.101:6443 check
server master02 10.50.0.102:6443 check
server master03 10.50.0.103:6443 check
十一、Keepalived VIP
作用：
提供：
HAProxy VIP
DNS VIP
Nginx VIP
架构
VIP
|
+-------+-------+
| |
MASTER BACKUP
Keepalived Keepalived
Keepalived配置
master:
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass password
}
virtual_ipaddress {
10.50.0.100
}
}
backup:
priority 90
十二、OpenVPN
功能：
- 员工远程接入
- LDAP认证
- MFA
架构：
User
|
VPN
|
OpenVPN
|
LDAP
|
Network
OpenVPN Compose
services:
openvpn:
image:
kylemanna/openvpn
ports:
- "1194:1194/udp"
cap_add:
- NET_ADMIN
LDAP认证
配置：
plugin /usr/lib/openvpn/openvpn-auth-ldap.so
<LDAP>
URL ldap://ldap.company.local
BindDN cn=admin,dc=company,dc=local
BaseDN ou=users,dc=company,dc=local
</LDAP>
十三、WireGuard
轻量VPN：
Client
|
WireGuard
|
Internal Network
配置：
server:
[Interface]
Address=10.8.0.1/24
ListenPort=51820
PrivateKey=xxx
client:
[Peer]
Endpoint=vpn.company.com:51820
AllowedIPs=10.50.0.0/16
十四、FTP / SFTP
SFTP 推荐
替代FTP：
- 加密
- SSH协议
- 易审计
SFTP Docker
services:
sftp:
image:
atmoz/sftp
ports:
- "2222:22"
volumes:
- /data/upload:/home/user/upload
command:
user:password:::upload
十五、vsftpd FTP
Compose：
services:
ftp:
image:
fauria/vsftpd
ports:
- "21:21"
environment:
FTP_USER:
ftpuser
FTP_PASS:
password
十六、防火墙
推荐：
Ubuntu:
nftables
CentOS:
firewalld
nftables规则
文件：
/etc/nftables.conf
示例：
table inet filter {
chain input {
type filter hook input priority filter;
policy drop;
ct state established,related accept
iif lo accept
tcp dport 22 accept
tcp dport 443 accept
tcp dport 6443 accept
}
}
十七、Kubernetes Ingress HA
生产：
Internet
|
HAProxy
|
Ingress Controller
|
Service
Ingress-nginx:
replicas: 2
配置：
controller:
replicaCount:
2
service:
type:
LoadBalancer
十八、网络检测
Ansible：
roles/network-check
tasks:
---
- name:
check dns
command:
nslookup kubernetes.default
- name:
check port
wait_for:
host:
"{{ inventory_hostname }}"
port:
443
十九、统一部署入口
DNS
ansible-playbook dns.yml
Proxy
ansible-playbook proxy.yml
HAProxy
ansible-playbook haproxy.yml
Keepalived
ansible-playbook keepalived.yml
VPN
ansible-playbook vpn.yml
Firewall
ansible-playbook firewall.yml
二十、生产推荐网络规划
示例：
| 用途 | 网段 |
|---|---|
| Management | 10.50.0.0/24 |
| Kubernetes | 10.50.10.0/24 |
| Storage | 10.50.20.0/24 |
| Database | 10.50.30.0/24 |
| VPN | 10.50.40.0/24 |
| Backup | 10.50.50.0/24 |
Part 10：Storage & Data Platform 存储数据层
本部分建设企业级存储与数据平台
目标：
- 分布式块存储
- 分布式文件存储
- 对象存储
- Kubernetes CSI存储
- 数据库高可用
- 消息队列
- 大数据存储
- 备份存储
- AI/LLM数据存储
Part 10 总体架构
Storage Platform
|
+----------------------+----------------------+
| |
Distributed Storage Database Platform
| |
| |
+------+-------+ +---------+---------+
| | | |
Ceph SeaweedFS PostgreSQL HA MySQL HA
| | |
| | |
RBD S3 Gateway Redis Cluster
|
|
CephFS
|
|
Kubernetes CSI
|
|
+------+---------+
| |
StatefulSet PVC
一、目录增加
enterprise-platform-ansible/
roles/
├── ceph/
├── minio/
├── seaweedfs/
├── nfs/
├── iscsi/
├── longhorn/
├── postgresql-ha/
├── mysql-ha/
├── mongodb/
├── redis/
└── kafka/
playbooks/
├── ceph.yml
├── minio.yml
├── seaweedfs.yml
├── database.yml
├── kafka.yml
└── storage-class.yml
二、存储规划
企业推荐：
| 类型 | 组件 | 用途 |
|---|---|---|
| Block Storage | Ceph RBD | 数据库/K8s PVC |
| File Storage | CephFS | 共享文件 |
| Object Storage | MinIO | AI/备份/S3 |
| Large File | SeaweedFS | 视频/LLM数据 |
| NAS | NFS | 简单共享 |
| Cloud Native | Longhorn | K8s存储 |
| Archive | Restic/MinIO | 备份 |
三、Ceph分布式存储
架构
Ceph Cluster
MON
+---------+---------+
| |
MON01 MON02
|
OSD
+--------+--------+
| | |
OSD01 OSD02 OSD03
|
RADOS
|
+--------+---------+
| |
RBD CephFS
四、Ceph节点规划
示例：
ceph01
MON
MGR
OSD
ceph02
MON
OSD
ceph03
MON
OSD
五、Ceph Ansible安装
目录：
roles/ceph/tasks
main.yml
---
- name:
install cephadm
shell:
curl --silent \
--remote-name \
https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm
- name:
install ceph
shell:
./cephadm add-repo \
--release reef
- name:
bootstrap ceph
shell:
cephadm bootstrap \
--mon-ip {{ ceph_mon_ip }}
六、Ceph Pool
创建RBD：
ceph osd pool create kubernetes
rbd pool init kubernetes
创建CephFS：
ceph fs volume create cephfs
七、Kubernetes Ceph CSI
安装：
helm repo add ceph-csi \
https://ceph.github.io/csi-charts
安装：
helm install ceph-csi \
ceph-csi/ceph-csi-rbd \
-n kube-system
StorageClass
apiVersion:
storage.k8s.io/v1
kind:
StorageClass
metadata:
name:
ceph-rbd
provisioner:
rbd.csi.ceph.com
parameters:
clusterID:
ceph
pool:
kubernetes
八、MinIO对象存储
用途：
- AI模型
- LLM知识库
- Backup
- Thanos
- Velero
- Harbor
MinIO架构
MinIO
+------+------+
| |
Node01 Node02
| |
Disk Pool
|
Erasure Coding
MinIO Docker Compose
services:
minio:
image:
minio/minio
command:
server \
/data{1...4} \
--console-address ":9001"
ports:
- "9000:9000"
- "9001:9001"
volumes:
- /data/minio1:/data1
- /data/minio2:/data2
- /data/minio3:/data3
- /data/minio4:/data4
environment:
MINIO_ROOT_USER:
admin
MINIO_ROOT_PASSWORD:
password
MinIO Bucket规划
minio
├── backup
│
├── velero
├── thanos
├── llm
├── model
├── file-storage
└── logs
MinIO Ansible
---
- name:
deploy minio
community.docker.docker_compose_v2:
project_src:
/opt/minio
九、SeaweedFS
适合：
- 海量小文件
- AI数据
- 图片
- 文档
SeaweedFS架构
SeaweedFS
|
Master
|
+---------+---------+
| |
Volume01 Volume02
|
Filer
|
S3 Gateway
十、SeaweedFS部署
Master
weed master \
-ip=seaweed01 \
-port=9333
Volume
weed volume \
-server=seaweed01:9333 \
-dir=/data/volume1 \
-port=8080
Filer
weed filer \
-master=seaweed01:9333
S3 Gateway
weed s3 \
-filer=seaweed01:8888
SeaweedFS Systemd
[Service]
ExecStart=/usr/local/bin/weed filer \
-master=seaweed01:9333
Restart=always
十一、NFS共享存储
适合：
- Jenkins workspace
- 软件包
- 小规模共享
服务端：
apt install nfs-server
exports:
/data/share
10.50.0.0/16(rw,sync,no_root_squash)
客户端：
mount -t nfs \
10.50.0.10:/data/share \
/mnt/share
十二、iSCSI块存储
用途：
- VMware
- Proxmox
- Database
安装：
apt install targetcli-fb
创建Target：
targetcli
十三、Longhorn Kubernetes Storage
适合：
- Kubernetes轻量存储
安装：
helm repo add longhorn \
https://charts.longhorn.io
helm install longhorn \
longhorn/longhorn \
-n longhorn-system
十四、PostgreSQL HA
架构：
HAProxy
|
+---------+---------+
| |
PostgreSQL01 PostgreSQL02
|
Patroni
|
etcd
Patroni部署
组件：
PostgreSQL
+
Patroni
+
etcd
+
HAProxy
PostgreSQL Docker
services:
postgres:
image:
postgres:16
environment:
POSTGRES_PASSWORD:
password
volumes:
- /data/postgres:/var/lib/postgresql/data
十五、MySQL HA
推荐：
MySQL InnoDB Cluster
+
MySQL Router
架构：
Router
|
+-----------+-----------+
| | |
MySQL01 MySQL02 MySQL03
Group Replication
十六、MongoDB ReplicaSet
架构：
MongoDB01
MongoDB02
MongoDB03
|
ReplicaSet
启动：
mongod \
--replSet rs0
初始化：
rs.initiate()
十七、Redis Cluster
架构：
Redis01
Redis02
Redis03
+
Sentinel
+
Replication
Redis Compose
services:
redis:
image:
redis:7
command:
redis-server \
--appendonly yes
十八、Kafka消息平台
用途：
- 日志
- AI Pipeline
- 数据流
Kafka架构
Producer
|
Kafka Broker
|
Consumer
Kafka KRaft模式
无需Zookeeper
Compose:
services:
kafka:
image:
bitnami/kafka
environment:
KAFKA_ENABLE_KRAFT:
yes
KAFKA_CFG_NODE_ID:
1
十九、数据库备份集成
统一进入：
Restic
+
MinIO
+
Velero
流程：
Database
|
Dump
|
Restic
|
MinIO
|
Archive
二十、Storage部署入口
Ceph
ansible-playbook ceph.yml
MinIO
ansible-playbook minio.yml
SeaweedFS
ansible-playbook seaweedfs.yml
Database
ansible-playbook database.yml
Kafka
ansible-playbook kafka.yml
二十一、最终企业存储拓扑
Application
|
Kubernetes
|
+----------+----------+
| |
CSI Storage Object Storage
| |
Ceph MinIO
|
CephFS/RBD
|
Database
PostgreSQL
MySQL
MongoDB
Redis
Kafka
|
Backup
Velero
Restic
Part 11：CI/CD & GitOps 持续交付层
本部分建设企业级 DevOps 自动化交付体系。
目标：
- Git代码管理
- 自动构建
- 自动测试
- 镜像构建
- 镜像安全扫描
- 制品管理
- Kubernetes自动发布
- GitOps持续部署
- 灰度发布
- 蓝绿发布
- DevSecOps安全流水线
Part 11 总体架构
Developer
|
Git Push
|
+----------------+
| Git Repository |
+----------------+
|
+------------------+------------------+
| |
GitLab CI Jenkins
| |
+------------------+------------------+
|
Build
|
Docker Image Build
|
Harbor
|
+--------------+--------------+
| |
Trivy SonarQube
Vulnerability Scan Code Quality
|
|
Security Gate
|
GitOps Repository
|
ArgoCD
|
Kubernetes Cluster
|
Argo Rollouts
|
Canary / Blue Green
一、目录增加
enterprise-platform-ansible/
roles/
├── gitlab-runner/
├── jenkins-agent/
├── sonarqube/
├── harbor/
├── argocd/
├── argocd-image-updater/
├── argo-rollouts/
├── tekton/
├── helm/
├── kustomize/
├── trivy-ci/
└── sbom/
playbooks/
├── cicd.yml
├── gitlab-runner.yml
├── jenkins-agent.yml
├── sonarqube.yml
├── argocd.yml
├── tekton.yml
└── security-pipeline.yml
二、DevOps平台组件
| 组件 | 用途 |
|---|---|
| GitLab | 代码仓库 |
| Jenkins | 流水线 |
| GitLab Runner | CI执行器 |
| Harbor | 镜像仓库 |
| SonarQube | 代码质量 |
| Trivy | 漏洞扫描 |
| Syft | SBOM |
| ArgoCD | GitOps |
| Argo Rollouts | 灰度发布 |
| Helm | 包管理 |
| Kustomize | 配置管理 |
| Tekton | 云原生CI |
三、GitLab CI/CD
Pipeline流程
.gitlab-ci.yml
|
|
Stage 1
Build
|
Stage 2
Test
|
Stage 3
Security
|
Stage 4
Docker Build
|
Stage 5
Push Harbor
|
Stage 6
Update GitOps
四、GitLab Runner部署
Docker Compose
目录：
docker-compose/gitlab-runner
docker-compose.yml
services:
gitlab-runner:
image:
gitlab/gitlab-runner:latest
container_name:
gitlab-runner
restart:
always
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /data/gitlab-runner:/etc/gitlab-runner
注册Runner
docker exec -it gitlab-runner \
gitlab-runner register
输入：
GitLab URL:
https://gitlab.company.com
Token:
xxxxx
Executor:
docker
Image:
docker:latest
Runner Ansible
roles/gitlab-runner/tasks/main.yml
---
- name:
deploy gitlab runner
community.docker.docker_compose_v2:
project_src:
/opt/gitlab-runner
五、GitLab CI模板
项目：
.gitlab-ci.yml
示例：
stages:
- build
- test
- security
- docker
build:
stage:
build
script:
- npm install
- npm run build
test:
stage:
test
script:
- npm test
security:
stage:
security
script:
- trivy fs .
docker:
stage:
docker
script:
- docker build -t harbor.company.com/app:v1 .
- docker push harbor.company.com/app:v1
六、Jenkins Pipeline
适合：
- Java
- 大型项目
- 多阶段发布
Jenkinsfile
pipeline {
agent any
stages {
stage('Checkout'){
steps{
git 'https://git.company.com/app.git'
}
}
stage('Build'){
steps{
sh 'mvn clean package'
}
}
stage('Docker'){
steps{
sh '''
docker build \
-t harbor.company.com/app:${BUILD_NUMBER} .
'''
}
}
stage('Deploy'){
steps{
sh '''
kubectl apply -f deploy.yaml
'''
}
}
}
}
七、Jenkins Kubernetes Agent
架构：
Jenkins
|
Kubernetes Plugin
|
Dynamic Pod
|
Build Container
Pod Template:
apiVersion:
v1
kind:
Pod
spec:
containers:
- name:
builder
image:
maven:3.9-jdk17
八、Harbor镜像仓库
作用：
Docker Image
+
Security Scan
+
Replication
+
RBAC
Harbor架构
Developer
|
GitLab CI
|
Docker Build
|
Harbor
|
Trivy
|
Kubernetes
Harbor自动部署
helm repo add harbor \
https://helm.goharbor.io
helm install harbor \
harbor/harbor \
-n harbor
九、SonarQube代码质量
检测：
- Bug
- Vulnerability
- Code Smell
- Coverage
Docker Compose
services:
sonarqube:
image:
sonarqube:lts
ports:
- "9000:9000"
environment:
SONAR_ES_BOOTSTRAP_CHECKS_DISABLE:
true
Sonar Maven集成
<plugin>
<groupId>
org.sonarsource.scanner.maven
</groupId>
<artifactId>
sonar-maven-plugin
</artifactId>
</plugin>
执行：
mvn sonar:sonar \
-Dsonar.host.url=http://sonar:9000
十、Trivy DevSecOps
扫描阶段：
Code
|
Build
|
Image
|
Trivy
|
PASS
|
Deploy
GitLab CI：
trivy:
stage:
security
script:
- trivy image \
harbor.company.com/app:$CI_COMMIT_SHA
十一、SBOM软件供应链安全
工具：
Syft
+
Grype
生成SBOM：
syft nginx:latest \
-o cyclonedx-json
输出：
sbom.json
漏洞扫描：
grype sbom.json
十二、ArgoCD GitOps
核心思想：
Git = Single Source Of Truth
Git修改
|
ArgoCD
|
Kubernetes
ArgoCD安装
helm repo add argo \
https://argoproj.github.io/argo-helm
helm install argocd \
argo/argo-cd \
-n argocd \
--create-namespace
ArgoCD目录结构
推荐：
gitops-repo/
├── applications/
├── charts/
├── environments/
│
├── dev/
├── test/
└── prod/
十三、Application定义
文件：
app.yaml
apiVersion:
argoproj.io/v1alpha1
kind:
Application
metadata:
name:
app-prod
spec:
project:
default
source:
repoURL:
https://git.company.com/gitops.git
path:
apps/app
destination:
server:
https://kubernetes.default.svc
namespace:
production
syncPolicy:
automated:
prune:
true
selfHeal:
true
十四、ArgoCD Image Updater
自动更新镜像：
Harbor
|
New Image
|
Image Updater
|
Git Commit
|
ArgoCD Sync
安装：
helm install argocd-image-updater \
argo/argocd-image-updater \
-n argocd
十五、Argo Rollouts 灰度发布
支持：
- Canary
- Blue Green
安装：
kubectl create namespace argo-rollouts
kubectl apply \
-f https://github.com/argoproj/argo-rollouts/releases/latest/download/install.yaml
Canary示例
apiVersion:
argoproj.io/v1alpha1
kind:
Rollout
spec:
strategy:
canary:
steps:
- setWeight:
10
- pause:
duration:
5m
- setWeight:
50
十六、Tekton Cloud Native CI
架构：
Git
|
Tekton Trigger
|
Pipeline
|
Task
|
Container Build
安装：
kubectl apply \
-f https://storage.googleapis.com/tekton-releases/pipeline/latest/release.yaml
十七、Helm规范
目录：
charts/
app/
├── Chart.yaml
├── values.yaml
└── templates/
Chart.yaml
apiVersion:
v2
name:
app
version:
1.0.0
values.yaml
image:
repository:
harbor.company.com/app
tag:
latest
replicas:
3
十八、Kustomize环境管理
目录：
overlays/
├── dev/
├── test/
└── prod/
prod/kustomization.yaml
resources:
- ../../base
patches:
- replica.yaml
十九、完整企业流水线
最终：
Developer
|
GitLab
|
CI Pipeline
|
+----------------+
| Build |
| Test |
| Sonar |
| Trivy |
| SBOM |
+----------------+
|
|
Harbor Registry
|
|
GitOps Repo
|
|
ArgoCD
|
|
Kubernetes
|
|
Canary Release
二十、CI/CD部署入口
GitLab Runner
ansible-playbook gitlab-runner.yml
Jenkins Agent
ansible-playbook jenkins-agent.yml
SonarQube
ansible-playbook sonarqube.yml
ArgoCD
ansible-playbook argocd.yml
Security Pipeline
ansible-playbook security-pipeline.yml
Part 12：AI Platform & MLOps 人工智能平台层
本部分建设企业级 AI 基础平台。
目标：
- GPU资源池化
- Kubernetes GPU调度
- 大模型推理服务
- 模型训练平台
- MLOps流程
- RAG知识库
- 向量数据库
- AI数据湖
- 模型生命周期管理
- AI应用发布体系
Part 12 总体架构
AI Platform
|
Kubernetes Cluster
|
+--------------------+--------------------+
| |
GPU Platform Data Platform
| |
NVIDIA GPU Operator MinIO / SeaweedFS
CUDA Toolkit PostgreSQL
TensorRT Kafka
|
|
+-------+---------+
| |
Inference Training
| |
vLLM Kubeflow
Ollama MLflow
TensorRT-LLM Airflow
|
|
AI Application
|
+-------+---------+
| |
RAG Platform Agent Platform
Milvus LangChain
ES LlamaIndex
一、目录增加
enterprise-platform-ansible/
roles/
├── nvidia-driver/
├── cuda/
├── nvidia-container-toolkit/
├── gpu-operator/
├── vllm/
├── ollama/
├── tensorrt-llm/
├── mlflow/
├── kubeflow/
├── airflow/
├── milvus/
├── elasticsearch-ai/
├── langchain/
├── llamaindex/
└── ai-gateway/
playbooks/
├── gpu-node.yml
├── gpu-operator.yml
├── inference.yml
├── mlops.yml
├── rag.yml
└── ai-platform.yml
二、GPU基础环境
GPU节点规划
示例：
gpu-node01
RTX PRO 5000
gpu-node02
RTX 4080S 32G
gpu-node03
A800/H800
三、NVIDIA Driver自动安装
支持：
- Ubuntu 22.04
- Ubuntu 24.04
- Rocky Linux
Ansible:
roles/nvidia-driver/tasks/main.yml
---
- name:
install nvidia driver
apt:
name:
- nvidia-driver-595
state:
present
- name:
reboot gpu node
reboot:
验证：
nvidia-smi
输出：
GPU
Driver Version
CUDA Version
Memory
四、CUDA Toolkit
安装：
apt install cuda-toolkit-12-8
验证：
nvcc --version
五、NVIDIA Container Toolkit
作用：
让Docker/Kubernetes使用GPU。
安装：
apt install nvidia-container-toolkit
配置：
nvidia-ctk runtime configure \
--runtime=docker
重启:
systemctl restart docker
测试：
docker run \
--rm \
--gpus all \
nvidia/cuda:12.8.0-base \
nvidia-smi
六、Kubernetes GPU支持
架构：
Kubernetes
|
NVIDIA Device Plugin
|
GPU Node
|
NVIDIA Driver
七、NVIDIA GPU Operator
安装：
helm repo add nvidia \
https://helm.ngc.nvidia.com/nvidia
helm install gpu-operator \
nvidia/gpu-operator \
-n gpu-operator \
--create-namespace
检查：
kubectl get pods \
-n gpu-operator
输出：
gpu-operator
nvidia-device-plugin
nvidia-driver
八、GPU节点标签
kubectl label node gpu01 \
accelerator=nvidia
九、GPU资源测试
Pod:
apiVersion: v1
kind: Pod
metadata:
name: gpu-test
spec:
containers:
- name:
cuda
image:
nvidia/cuda:12.8-base
resources:
limits:
nvidia.com/gpu:
1
验证：
kubectl exec gpu-test -- nvidia-smi
十、大模型推理平台
包含：
| 组件 | 用途 |
|---|---|
| vLLM | 高性能LLM推理 |
| TensorRT-LLM | NVIDIA优化 |
| Ollama | 轻量模型运行 |
| TGI | HuggingFace推理 |
十一、vLLM部署
架构：
Client
|
API Gateway
|
vLLM Server
|
GPU
Docker Compose:
services:
vllm:
image:
vllm/vllm-openai
runtime:
nvidia
volumes:
- /data/models:/models
command:
--model /models/Qwen2.5-72B
ports:
- 8000:8000
启动：
docker compose up -d
API测试：
curl http://localhost:8000/v1/models
Chat API:
curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '
{
"model":"Qwen2.5",
"messages":[
{"role":"user","content":"hello"}
]
}
'
十二、Ollama平台
适合：
- 本地AI
- 开发测试
- 小模型
Compose:
services:
ollama:
image:
ollama/ollama
runtime:
nvidia
volumes:
- /data/ollama:/root/.ollama
ports:
- 11434:11434
运行：
ollama run qwen2.5
十三、TensorRT-LLM
用途：
NVIDIA高性能推理。
架构：
Model
|
TensorRT Build
|
Engine
|
GPU Runtime
转换：
trtllm-build \
--checkpoint_dir model \
--output_dir engine
十四、模型仓库
推荐：
MinIO
目录：
models/
├── qwen/
├── llama/
├── embedding/
├── reranker/
└── whisper/
十五、MLflow模型管理
功能：
- 实验记录
- 模型版本
- 模型注册
架构：
Training
|
MLflow
|
Model Registry
|
Production
MLflow Compose
services:
mlflow:
image:
ghcr.io/mlflow/mlflow
ports:
- 5000:5000
command:
mlflow server
--host 0.0.0.0
访问：
http://mlflow.company.com
十六、Kubeflow
用于：
- AI训练流水线
- Notebook
- Pipeline
架构：
Kubeflow
├── Notebook
├── Training Operator
├── Pipeline
├── Katib
└── Model Serving
安装：
export PIPELINE_VERSION=2.3.0
kubectl apply \
-k https://github.com/kubeflow/manifests
十七、Airflow
用途：
AI数据任务调度。
架构：
Scheduler
|
Worker
|
Task
Compose:
services:
airflow:
image:
apache/airflow
ports:
- 8080:8080
DAG示例：
from airflow import DAG
from airflow.operators.python import PythonOperator
def train():
print("training")
with DAG(
"ai-training"
) as dag:
task=PythonOperator(
task_id="train",
python_callable=train
)
十八、RAG平台
架构：
Document
|
Parser
|
Embedding Model
|
Vector Database
|
Retriever
|
LLM
|
Answer
十九、Milvus向量数据库
用途：
- 文档检索
- 知识库
- Agent Memory
Compose:
services:
milvus:
image:
milvusdb/milvus
ports:
- 19530:19530
Collection:
knowledge_base
fields:
id
embedding
text
metadata
二十、Elasticsearch AI搜索
用途：
- 全文检索
- 混合搜索
架构：
Keyword Search
+
Vector Search
=
Hybrid Search
索引：
{
"mappings":{
"properties":{
"text":{
"type":"text"
},
"vector":{
"type":"dense_vector"
}
}
}
}
二十一、LangChain
用途：
AI应用开发框架。
架构：
Application
|
LangChain
|
Retriever
|
LLM
|
Tools
安装：
pip install langchain
示例：
from langchain_openai import ChatOpenAI
llm=ChatOpenAI(
base_url="http://vllm:8000"
)
response=llm.invoke(
"hello"
)
二十二、LlamaIndex
用途：
数据连接层。
支持：
- MySQL
- Elasticsearch
- Milvus
- MinIO
安装：
pip install llama-index
二十三、AI Gateway
统一入口：
User
|
API Gateway
|
+--------------+
| |
OpenAI API Local LLM
vLLM Ollama
推荐：
- Higress
- Kong
- APISIX
二十四、AI安全
增加：
Prompt防护
Prompt Injection
Sensitive Data
Jailbreak
模型权限
Keycloak:
User
|
OAuth2
|
AI API
审计
记录：
user
model
prompt
response
token
cost
进入：
Loki
+
Grafana
二十五、AI流水线
完整流程：
Data
|
SeaweedFS
|
Airflow
|
Feature Engineering
|
Kubeflow
|
Training
|
MLflow
|
Model Registry
|
Harbor
|
vLLM
|
API Gateway
|
Application
二十六、Ansible部署入口
GPU：
ansible-playbook gpu-node.yml
GPU Operator：
ansible-playbook gpu-operator.yml
推理平台：
ansible-playbook inference.yml
MLOps：
ansible-playbook mlops.yml
RAG：
ansible-playbook rag.yml
完整AI平台：
ansible-playbook ai-platform.yml
Part 13：Enterprise Automation & Operations 运维自动化层
本部分建设企业级 IT 自动化运维平台。
目标：
- 基础设施自动化
- 自动装机
- 配置管理
- 云资源编排
- CMDB资产管理
- ITSM流程管理
- 自动巡检
- 自动修复
- ChatOps运维机器人
- 企业级运维闭环
Part 13 总体架构
Enterprise Operations Platform
|
+-----------+-----------+
| |
Automation Engine Infrastructure
| |
+-----------+-----------+ |
| | |
Ansible AWX Terraform PXE
Rundeck OpenTofu Kickstart
Jenkins Packer Cloud-init
|
+------------+-------------+
| |
CMDB / ITSM Monitoring
| |
Assets Prometheus
Tickets Grafana
Workflow Alertmanager
|
ChatOps Robot
|
+-----------+-----------+
|
Auto Remediation
一、目录增加
enterprise-platform-ansible/
roles/
├── awx/
├── ansible-runner/
├── rundeck/
├── terraform/
├── opentofu/
├── packer/
├── pxe/
├── kickstart/
├── cloud-init/
├── cmdb/
├── itsm/
├── chatops/
├── auto-remediation/
├── bastion/
└── operation-agent/
playbooks/
├── automation.yml
├── awx.yml
├── rundeck.yml
├── terraform.yml
├── pxe.yml
├── cmdb.yml
├── chatops.yml
└── remediation.yml
二、自动化平台选型
| 功能 | 组件 |
|---|---|
| 配置管理 | Ansible |
| Web自动化 | AWX |
| 任务调度 | Rundeck |
| 基础设施编排 | Terraform |
| Terraform替代 | OpenTofu |
| 镜像制作 | Packer |
| 裸机安装 | PXE |
| 初始化 | Cloud-init |
| 资产管理 | CMDB |
| 流程管理 | ITSM |
| 机器人 | ChatOps |
| 自动修复 | Ansible + Alertmanager |
三、Ansible Automation Platform
架构
Developer
|
Git Repository
|
Ansible
|
+-------+-------+
| |
AWX UI CLI
|
Execution Node
|
Linux / Network / Cloud
四、AWX部署
推荐 Kubernetes 部署。
架构：
Kubernetes
|
AWX Operator
|
+---------+---------+
| |
AWX PostgreSQL
安装AWX Operator
kubectl create namespace awx
kubectl apply \
-f https://github.com/ansible/awx-operator/releases/latest/download/awx-operator.yaml
AWX实例
文件：
awx.yaml
apiVersion:
awx.ansible.com/v1beta1
kind:
AWX
metadata:
name:
awx
spec:
service_type:
NodePort
postgres_storage_class:
ceph-rbd
部署：
kubectl apply -f awx.yaml
五、AWX组织结构
推荐：
AWX
├── Organizations
│
├── Production
│
├── Development
├── Projects
│
├── Network
│
├── Kubernetes
├── Inventories
│
├── Prod
│
├── Test
├── Job Templates
└── Credentials
六、Ansible Inventory设计
目录:
inventory/
├── production
│
├── hosts.yml
├── group_vars/
└── host_vars/
示例：
all:
children:
kubernetes:
hosts:
master01:
ansible_host:
10.50.0.101
worker01:
ansible_host:
10.50.0.111
database:
hosts:
pg01:
ansible_host:
10.50.0.50
七、Rundeck任务平台
用途：
- 运维人员自助执行
- 定时任务
- 权限控制
架构：
User
|
Rundeck
|
Ansible
|
Server
Docker Compose
services:
rundeck:
image:
rundeck/rundeck
ports:
- "4440:4440"
volumes:
- /data/rundeck:/home/rundeck/server/data
Rundeck Job示例
- defaultTab: nodes
description:
Restart nginx
sequence:
commands:
- exec:
sudo systemctl restart nginx
八、Terraform基础设施自动化
用途：
- VMware
- OpenStack
- AWS
- Azure
- Proxmox
目录：
terraform/
├── main.tf
├── variables.tf
├── outputs.tf
└── modules/
示例：创建VM
resource "proxmox_vm_qemu" "node01" {
name="k8s-node01"
cores=8
memory=32768
clone="ubuntu2404"
}
执行：
terraform init
terraform plan
terraform apply
九、OpenTofu
Terraform开源分支。
安装：
apt install opentofu
执行：
tofu init
tofu apply
十、Packer镜像自动构建
用途：
自动生成：
- Ubuntu镜像
- Rocky镜像
- Kubernetes Node镜像
流程：
ISO
|
Packer
|
VM Template
|
Terraform
|
Production VM
示例：
source "vmware-iso" "ubuntu" {
iso_url="ubuntu.iso"
ssh_username="root"
}
build {
sources=[
"source.vmware-iso.ubuntu"
]
}
十一、PXE自动装机平台
用途：
批量安装服务器。
架构：
Server
|
PXE Boot
|
DHCP
|
TFTP
|
HTTP
|
Kickstart/Autoinstall
|
OS
组件：
DHCP
TFTP
HTTP
DNS
iPXE
十二、PXE Docker Compose
services:
dnsmasq:
image:
jpillora/dnsmasq
ports:
- "67:67/udp"
- "69:69/udp"
volumes:
- ./conf:/etc/dnsmasq.conf
十三、Ubuntu 24.04 Autoinstall
文件：
user-data
示例：
autoinstall:
version:
1
identity:
hostname:
node01
username:
ubuntu
password:
encrypted-password
storage:
layout:
name:
lvm
十四、Cloud-init
用途：
云初始化
支持：
- OpenStack
- VMware
- Proxmox
- Public Cloud
cloud-init:
#cloud-config
hostname:
app01
packages:
- docker.io
- git
runcmd:
- systemctl enable docker
十五、CMDB资产管理
推荐：
- NetBox
- Snipe-IT
- GLPI
NetBox架构
Device
|
NetBox
|
API
|
Automation
|
Ansible
NetBox Docker
services:
netbox:
image:
netboxcommunity/netbox
ports:
- "8000:8080"
NetBox数据模型
Tenant
|
Site
|
Rack
|
Device
|
Interface
|
IP Address
十六、ITSM流程
推荐：
- Jira Service Management
- ServiceNow
- GLPI
流程：
User
|
Ticket
|
Approval
|
Automation
|
Change
|
Audit
十七、ChatOps运维机器人
支持：
- 企业微信
- 钉钉
- Slack
- Teams
架构：
Engineer
|
Chat
|
Bot
|
Webhook
|
AWX/Rundeck
|
Execute Job
十八、机器人命令示例
用户：
重启订单服务
机器人：
检测权限
确认环境
调用AWX
执行:
kubectl rollout restart deployment order
返回结果
十九、Alertmanager自动修复
流程：
Prometheus
|
Alertmanager
|
Webhook
|
Robot
|
Ansible
|
Fix
Alert规则
groups:
- name:
node
rules:
- alert:
DiskFull
expr:
node_filesystem_avail_bytes < 10%
二十、自动修复Playbook
示例：
---
- name:
restart service
hosts:
all
tasks:
- name:
restart nginx
service:
name:
nginx
state:
restarted
二十一、Bastion跳板机
架构：
Engineer
|
VPN
|
Bastion
|
Production
安全：
- SSH审计
- MFA
- 命令记录
推荐：
- Teleport
- Boundary
- Jumpserver
二十二、运维Agent
部署：
Node
|
Node Exporter
|
OSQuery
|
Falco
|
Telegraf
二十三、统一运维流程
完整闭环：
资产入库
|
CMDB
|
自动安装
|
PXE
|
配置管理
|
Ansible
|
部署应用
|
GitOps
|
监控
|
告警
|
自动修复
|
工单关闭
二十四、自动化入口
AWX
ansible-playbook awx.yml
Rundeck
ansible-playbook rundeck.yml
Terraform
ansible-playbook terraform.yml
PXE
ansible-playbook pxe.yml
CMDB
ansible-playbook cmdb.yml
ChatOps
ansible-playbook chatops.yml
Auto Remediation
ansible-playbook remediation.yml
Part 14：Enterprise Architecture Integration & Production Deployment 企业架构集成与生产交付层
本部分将前面 Part 1 ～ Part 13 的所有组件整合成一个完整企业级生产平台
目标：
- 企业整体架构设计
- 多数据中心规划
- 双活/灾备体系
- 网络安全域规划
- Kubernetes生产架构
- 数据平台整合
- AI平台整合
- 自动化交付体系
- Day0-Day2 运维流程
- 一键化部署入口
- 企业交付文档模板
一、完整企业平台最终架构
Internet
|
Edge Firewall
|
+----------------+
| Load Balance |
| HAProxy |
+----------------+
|
+----------------+
| API Gateway |
| Higress/APISIX |
+----------------+
|
+-------------------------+-------------------------+
| |
Kubernetes Production Cluster Enterprise Services
| |
+-------+--------+ +----------+----------+
| | | |
Application AI Platform DevOps Platform Office
| | | |
Microservice vLLM GitLab Jira
Java/Python Ollama Jenkins Confluence
Go MLflow Harbor Nextcloud
|
|
Storage Layer
|
+-------+---------+
| |
Ceph MinIO
RBD S3
CephFS Backup
|
Database Layer
|
+-------+---------+
| | |
PG HA MySQL MongoDB
Redis Kafka
|
Monitoring
|
Prometheus
Grafana
Loki
Thanos
|
Security
|
Vault
Keycloak
LDAP
Falco
Trivy
二、生产环境分层设计
Layer 0：基础设施层
包含：
服务器
网络
存储
机房
电源
IP
DNS
Layer 1：OS层
组件：
Ubuntu 24.04
Rocky Linux 9
NVIDIA Driver
Kernel
NTP
SSH Hardening
Audit
Layer 2：基础服务层
DNS
LDAP
Proxy
SMTP
FTP
VPN
NTP
Layer 3：DevOps层
GitLab
Gerrit
Jenkins
Nexus
Harbor
SonarQube
Layer 4：Office协作
Jira
Confluence
Nextcloud
Collabora
OnlyOffice
Mail
Layer 5：Kubernetes
Control Plane
Worker
Ingress
CSI
CNI
RBAC
Operator
Layer 6：数据平台
Ceph
MinIO
SeaweedFS
PostgreSQL
Kafka
MongoDB
Layer 7：AI平台
GPU
CUDA
vLLM
MLflow
Kubeflow
RAG
Layer 8：运维安全
Prometheus
Grafana
Vault
Keycloak
AWX
Terraform
三、多数据中心架构
生产推荐：
IDC-A IDC-B
Primary Disaster
| |
+---------------+ +---------------+
| Kubernetes | | Kubernetes |
| Storage | <------> | Storage |
| Database | | Database |
+---------------+ +---------------+
四、双活架构
适合：
- 金融
- 制造
- SaaS
架构：
DNS
|
+---------+---------+
| |
IDC1 IDC2
| |
Kubernetes Kubernetes
| |
Database Sync Database Sync
五、灾备架构
推荐：
生产
Primary
灾备
Warm Standby
数据同步：
| 数据 | 方案 |
|---|---|
| 对象存储 | MinIO Replication |
| 数据库 | Patroni/MySQL Replica |
| K8s资源 | Velero |
| 文件 | Restic |
| 配置 | GitOps |
六、服务器角色规划
示例：
Management节点
mgmt01
AWX
GitLab
Jenkins
Grafana
Kubernetes Master
k8s-master01
k8s-master02
k8s-master03
Worker
worker01
worker02
worker03
GPU Node
gpu01
gpu02
gpu03
Storage Node
storage01
storage02
storage03
七、IP规划
示例：
管理网
10.50.0.0/24
| 用途 | IP |
|---|---|
| DNS01 | 10.50.0.10 |
| DNS02 | 10.50.0.11 |
| HAProxy VIP | 10.50.0.100 |
| AWX | 10.50.0.20 |
Kubernetes
10.50.10.0/24
| 节点 | IP |
|---|---|
| master01 | 10.50.10.11 |
| master02 | 10.50.10.12 |
| master03 | 10.50.10.13 |
| worker01 | 10.50.10.21 |
Storage
10.50.20.0/24
| 服务 | IP |
|---|---|
| ceph01 | 10.50.20.11 |
| ceph02 | 10.50.20.12 |
| ceph03 | 10.50.20.13 |
八、域名规划
企业DNS：
company.com
基础服务：
git.company.com
jenkins.company.com
harbor.company.com
nexus.company.com
sonar.company.com
grafana.company.com
argocd.company.com
AI：
ai.company.com
llm.company.com
model.company.com
九、安全域划分
网络：
Firewall
|
+-----------------+-----------------+
| | |
DMZ APP DATA
DMZ
允许：
443
80
VPN
APP
允许：
8080
8443
DATA
禁止外部访问。
只允许：
APP --> DATA
十、Kubernetes生产设计
Master HA
3 Master
etcd HA
HAProxy VIP
Container Runtime
推荐：
containerd
CNI
推荐：
Calico
Cilium
Storage
Ceph CSI
Ingress
Higress
Ingress-Nginx
十一、GitOps生产流程
完整流程：
Developer
|
GitLab
|
CI
|
Build
|
Security Scan
|
Harbor
|
GitOps Repo
|
ArgoCD
|
Kubernetes
|
Production
十二、数据库生产架构
PostgreSQL
HAProxy
|
Patroni
|
PostgreSQL
|
Replication
MySQL
MySQL Router
|
InnoDB Cluster
|
Group Replication
Redis
Redis Sentinel
+
Replication
Kafka
Kafka Broker x3
KRaft
Replication Factor=3
十三、AI生产架构
User
|
AI Gateway
|
+---------+---------+
| |
RAG LLM
| |
Milvus vLLM
|
Embedding
|
MinIO
十四、监控体系
四大黄金指标：
Metrics
Prometheus
Logs
Loki
ELK
Tracing
Jaeger
Tempo
Visualization
Grafana
十五、自动化部署总入口
最终目录：
enterprise-platform-ansible/
├── inventory/
├── group_vars/
├── roles/
├── docker-compose/
├── helm/
├── kubernetes/
├── terraform/
├── playbooks/
└── README.md
十六、总安装入口
初始化服务器
ansible-playbook \
playbooks/bootstrap.yml
网络层
ansible-playbook \
playbooks/network.yml
Storage
ansible-playbook \
playbooks/storage.yml
Kubernetes
ansible-playbook \
playbooks/kubernetes.yml
DevOps
ansible-playbook \
playbooks/devops.yml
Monitoring
ansible-playbook \
playbooks/monitoring.yml
Security
ansible-playbook \
playbooks/security.yml
AI
ansible-playbook \
playbooks/ai-platform.yml
十七、一键安装总Playbook
文件：
site.yml
内容：
---
- import_playbook: bootstrap.yml
- import_playbook: network.yml
- import_playbook: storage.yml
- import_playbook: kubernetes.yml
- import_playbook: devops.yml
- import_playbook: office.yml
- import_playbook: monitoring.yml
- import_playbook: security.yml
- import_playbook: ai-platform.yml
- import_playbook: backup.yml
执行：
ansible-playbook site.yml
十八、Day0-Day2 运维流程
Day0 基础准备
服务器上架
网络配置
IP规划
DNS
PXE安装
Day1 平台部署
OS
Docker
Kubernetes
Storage
Database
Middleware
Day2 应用上线
GitLab
CI/CD
GitOps
Monitoring
Backup
Security
十九、生产验收Checklist
OS
✓ NTP
✓ SSH安全
✓ Kernel参数
✓ Audit
Kubernetes
✓ HA
✓ RBAC
✓ NetworkPolicy
✓ CSI
✓ Backup
Storage
✓ Ceph HEALTH_OK
✓ MinIO Replication
✓ Backup Test
Security
✓ Vault
✓ Keycloak
✓ LDAP
✓ Vulnerability Scan
Monitoring
✓ Metrics
✓ Logs
✓ Alert
✓ Dashboard
二十、企业交付文档结构
最终输出：
Enterprise-Platform-Documentation/
├── Architecture/
│
├── Network Design/
│
├── Security Design/
│
├── Kubernetes Design/
│
├── Storage Design/
│
├── Operation Manual/
│
├── Backup Manual/
│
├── Disaster Recovery/
│
├── Deployment Guide/
│
└── Troubleshooting/
至此完整企业平台蓝图：
Part 1 OS基础层
Part 2 基础服务层
Part 3 DevOps层
Part 4 Office层
Part 5 Kubernetes层
Part 6 Monitoring层
Part 7 Backup层
Part 8 Security层
Part 9 Network & HA层
Part 10 Storage & Data Platform层
Part 11 CI/CD & GitOps层
Part 12 AI Platform & MLOps层
Part 13 Enterprise Automation层
Part 14 Enterprise Architecture Integration层
477
扫一扫
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
