k8sGPT 加入 CNCF Sandbox Project!
去年開始 Cloud Native Computing Foundation 針對 AI 領域有開始著墨，在 2023/12/19 k8sGPT 正式被放入到 Sandbox Project 內，這專案目的是協助你一行 k8sgpt analyze --explain --anonymize 分析你現在登入的 Kubernetes 狀況，讓你擺脫觀落陰的輪迴中
有趣的事情是，他是被分類到 TAG Observability 範疇內...
k8sgpt 現行支援 Azure OpenAI
依據 LLM AI Backends 清單，除了 OpenAI 以外，也有支援 Azure OpenAI
k8sgpt-azureopenai-install.sh
#
# Prerequisites
#
AOAI_ENDPOINT="https://pichuang.openai.azure.com/"
DEPLOYMNET_NAME="deployment-gpt35-turbo-16k" # It's a deployment name, not a model name
MODEL_NAME="gpt-35-turbo-16k"
AOAI_API_KEY="pichuangishandsome"
#
# Add Azure OpenAI backend
#
$ k8sgpt auth add --backend azureopenai --baseurl $AOAI_ENDPOINT --engine $DEPLOYMNET_NAME --model $MODEL_NAME --password $AOAI_API_KEY
azureopenai added to the AI backend provider list
# Set Azure OpenAI backend as default
$ k8sgpt auth default -p azureopenai
Default provider set to azureopenai
# List all backends
$ k8sgpt auth list
Default:
> azureopenai
Active:
> azureopenai
# Check you can reach Kubernetes cluster, and here I use OpenShift Local v4.14.7
$ oc version
Client Version: 4.14.7
Kustomize Version: v5.0.1
Server Version: 4.14.7
Kubernetes Version: v1.27.8+4fab27b
$ kubectl version
Client Version: v1.29.1
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.27.8+4fab27b
$ kubectl cluster-info
Kubernetes control plane is running at https://api.crc.testing:6443
# Test k8sgpt with Azure OpenAI backend
$ k8sgpt analyze --explain --anonymize
100% |███████████████████████████████████████████████████████████████████████████████████████████| (3/3, 31 it/min)
AI Provider: azureopenai
...omit...
這邊我就拿我電腦上面有的 Red Hat OpenShift Local 來進行使用，當然你如果是用 Azure Kubernetes Service / Tanzu Kubernetes Grid 的也適用這個作法，k8sgpt 基本上就從 API 那邊撈東西回來後，幫你爬 Log 以外，還可以針對錯誤給一點 Solution 讓你有一個方向可以解決問題
[無關 k8sgpt] 提升查詢速度和資料隱蔽性
基於 Azure OpenAI - Use Private Endpoints，可以將傳輸速度和隱蔽性透過 Private Endpoint 的路線來大幅提升
下面的 VM 和 Azure OpenAI 及其 Private Endpoint 都在 East US 2，光在同區測試這樣就有差 0.5 ~ 1 個 ms 了
private-versus-public-routing
#
# via Public Internet (Microsoft Routing)
#
$ sudo hping3 -S pichuang.openai.azure.com -p 443 -c 5
HPING pichuang.openai.azure.com (eth0 20.119.156.137): S set, 40 headers + 0 data bytes
len=44 ip=20.119.156.137 ttl=58 DF id=0 sport=443 flags=SA seq=0 win=64240 rtt=3.7 ms
len=44 ip=20.119.156.137 ttl=56 DF id=0 sport=443 flags=SA seq=1 win=64240 rtt=3.6 ms
len=44 ip=20.119.156.137 ttl=58 DF id=0 sport=443 flags=SA seq=2 win=64240 rtt=3.5 ms
len=44 ip=20.119.156.137 ttl=58 DF id=0 sport=443 flags=SA seq=3 win=64240 rtt=3.4 ms
len=44 ip=20.119.156.137 ttl=56 DF id=0 sport=443 flags=SA seq=4 win=64240 rtt=3.3 ms
--- pichuang.openai.azure.com hping statistic ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 3.3/3.5/3.7 ms
#
# via Private Endpoint
#
$ sudo hping3 -S 10.0.0.9 -p 443 -c 5
HPING 10.0.0.9 (eth0 10.0.0.9): S set, 40 headers + 0 data bytes
len=44 ip=10.0.0.9 ttl=64 id=391 sport=443 flags=SA seq=0 win=64800 rtt=2.7 ms
len=44 ip=10.0.0.9 ttl=64 id=392 sport=443 flags=SA seq=1 win=64800 rtt=2.6 ms
len=44 ip=10.0.0.9 ttl=64 id=393 sport=443 flags=SA seq=2 win=64800 rtt=2.5 ms
len=44 ip=10.0.0.9 ttl=64 id=394 sport=443 flags=SA seq=3 win=64800 rtt=2.4 ms
len=44 ip=10.0.0.9 ttl=64 id=395 sport=443 flags=SA seq=4 win=64800 rtt=2.3 ms
--- 10.0.0.9 hping statistic ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 2.3/2.5/2.7 ms
因為 Azure OpenAI 是已 https 為主的服務，所以要正確使用，不跳過 TLS 驗證的話，建議要將 Azure Private DNS Resolver 和 Azure Private DNS 相關的架構弄好，才能正確運行
without-private-dns
#
# Inside VM with Azure Private Link
#
$ cat /etc/resolv.conf
# Generated by NetworkManager
search salqh4dnbine5ngi3moox0djkd.cx.internal.cloudapp.net divecode.intranet
nameserver 168.63.129.16
# Without Azure Private DNS
# Verify Azure Private DNS is working
#
$ dig pichuang.openai.azure.com
; <<>> DiG 9.16.23-RH <<>> pichuang.openai.azure.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 11405
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1224
;; QUESTION SECTION:
;pichuang.openai.azure.com. IN A
;; ANSWER SECTION:
pichuang.openai.azure.com. 900 IN CNAME pichuang.privatelink.openai.azure.com.
pichuang.privatelink.openai.azure.com. 900 IN CNAME eastus2.prod.vnet.cog.trafficmanager.net.
eastus2.prod.vnet.cog.trafficmanager.net. 60 IN CNAME vnetproxyv3-use2-prod.eastus2.cloudapp.azure.com.
vnetproxyv3-use2-prod.eastus2.cloudapp.azure.com. 10 IN A 20.119.156.137
;; Query time: 5 msec
;; SERVER: 168.63.129.16#53(168.63.129.16)
;; WHEN: Wed Jan 24 23:06:46 CST 2024
;; MSG SIZE rcvd: 210
with-private-dns
#
# After setting up Azure Private DNS and Private Link
# Verify Azure Private DNS is working
#
$ dig pichuang.openai.azure.com
; <<>> DiG 9.16.23-RH <<>> pichuang.openai.azure.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 20469
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1224
;; QUESTION SECTION:
;pichuang.openai.azure.com. IN A
;; ANSWER SECTION:
pichuang.openai.azure.com. 900 IN CNAME pichuang.privatelink.openai.azure.com.
pichuang.privatelink.openai.azure.com. 10 IN A 10.0.0.9
;; Query time: 3 msec
;; SERVER: 168.63.129.16#53(168.63.129.16)
;; WHEN: Wed Jan 24 21:37:47 CST 2024
;; MSG SIZE rcvd: 103
文後廢言
今天幫客戶弄 Microsoft Server 喬 DNS Forwarder，結果被看出來不是很熟 Windows Server 操作，被問怎麼進微軟的 (煙)，來微軟上班一年半，終於認真寫到一次 Powershell，只能慶幸不愧是微軟本家，Poweshell SDK 提供的算完整，搭個 GitHub Copilot 就被我糊裡糊塗地寫完了