Internal Service in Kubernetes Cluster

In Kubernetes(K8s) cluster, 1 or more containers form a pod and every container in the pod can access other container’s port just like apps in the same local host. For example:

- pod1
  - nginx1
  - gunicorn1, port:8000

- pod2
  - nginx2
  - gunicorn2, port:8000

So nginx1 can access gunicorn1’s port using localhost:8000 and nginx2 can access gunicorn2, etc… However nginx1 can’t see gunicorn2 using localhost.

When it comes to cache, like redis, I would like a shared redis container(or cluster later) so less memory is wasted and the cache doesn’t need to be warmed for each pod. The structure will look like:

- pod1
  - nginx1
  - gunicorn1
- pod2
  - nginx2
  - gunicorn2
- pod3
  - redis

To grant both gunicorns to access redis, redis needs to be a service:

---
apiVersion: v1
kind: Service
metadata:
  name: redis-svc
  labels:
    app: redis
    role: cache
spec:
  type: NodePort
  ports:
    - port: 6379
  selector:
    app: redis
    role: cache

And the deployment to support the service looks like:

---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: redis-deploy
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: redis
        role: cache
    spec:
      containers:
        - name: redis
          image: redis:4
          resources:
            requests:
              memory: 200Mi
          ports:
            - containerPort: 6379

Then in the settings.py of the django app running in gunicorn container, redis can be accessed via ENVs setup by K8s:

CACHES = {
  "default": {
    "BACKEND": "django_redis.cache.RedisCache",
    "LOCATION": "redis://{0}:{1}/1".format(os.environ['REDIS_SVC_SERVICE_HOST'], os.environ['REDIS_SVC_SERVICE_PORT']),
    "OPTIONS": {
      "CLIENT_CLASS": "django_redis.client.DefaultClient"
    },
  }
}

🙂

A Simple Python IO Trick

I thought using Python’s gzip is quite straight forward, however the IO performance almost doubled with an extra BufferedWriter:

import io
import gzip

def export_to_json_gz(schema):
  json_file = "test.json.gz"
  with io.BufferedWriter( gzip.open(temp_dir + json_file, 'wb') ) as gzfile:
    for row in stream_table_data(schema):
      ujson.dump(row, gzfile, ensure_ascii=False)
      gzfile.write('\n')

Now the bottle neck is ujson, how can I make that part faster?

🙂

学TurboGears 2顺便还得学习一下Genshi

Genshi是Python环境下的一个比较出名的HTML/XML/…模板框架. 作为TurboGears2.1缺省的模板组件, Genshi得到了很多好评. 我也正是因为TG粘合了如此优秀的组件而决心学习TG的.

TG2.1样本项目里的主模板文件master.html的完整解释: http://www.turbogears.org/2.1/docs/main/master_html.html#master-html
Genshi模板基础: http://genshi.edgewall.org/wiki/Documentation/templates.html

首先, 在TG2中通过@expose decorator来为一个View(V in MVC)指定Genshi模板. 如果不需要模板, 就是:

@expose()
def hello(self):
    return "Hello world from controller."

这样在缺省的开发环境下, 访问http://127.0.0.1:8080/hello, 将得到简单的

Hello world from controller.

当然, 这除了测试似乎没有什么实际意义. 然后是使用模板的版本:

    @expose('example.templates.index')
    def new_hello(self):
        return dict(hello="New text from controller + template ", page="new_hello")

这样, TG会寻找example项目下的example/templates/index.html作为new_hello的模板. 同时, 将两个key传递给模板: hello和page. 而在index模板内, 可以很随意的调用这两个获得的key.

<h1 py:content=”hello”>old Hello, world.</h1>
Now Viewing: <span py:replace=”page”/>

区别在于, <h1>内的内容”old Hello, world.”会被hello的值替代, 而<span>则并不会出现, 完全被page值取代. 另外模板里的${text} 和php里面的<?php echo $text; ?>类似吧.

 

学习Python + TurboGears 2

前一阵子我着实的困惑了几分钟: 下半辈子做什么呢? 当别人努力的stand out时, 我却在不停的试图blend in. 好在依旧自信, 我索性说, who cares, 我要继续学习我想学的东西: 互联网技术+摄影, 不管有没有搞头, 应该不至于挨饿. 废话到此为止.

选择TG而淘汰Django的原因是TG的耦合度较低, 而且学习难度要大些. 你了解我的, 我喜欢做难度大的事情, 例如投胎到中国. TG目前的版本是2.1, 安装方法可以参考:

http://www.turbogears.org/2.1/docs/main/DownloadInstall.html

在Ubuntu环境下, 先安装必要基础:

$ sudo aptitude install build-essential python-dev python-setuptools python-virtualenv

然后在CLI里如下方法开始第一个TG项目:

$ virtualenv –no-site-packages -p python2.6 tg2env
$ cd tg2env/
$ source bin/activate
(tg2env)$ easy_install -i http://www.turbogears.org/2.1/downloads/current/index tg.devtools
(tg2env)$ paster quickstart example
(tg2env)$ cd example/
(tg2env)$ python setup.py develop
(tg2env)$ nosetests
(tg2env)$ paster setup-app development.ini
(tg2env)$ paster serve development.ini
(tg2env)$ deactivate

其中的example可以换成自己的项目名称. 在运行了paster serve之后, 用浏览器访问http://127.0.0.1:8080/就出现题图的样子了. 🙂

等完成基本的MVC模式再写下一篇了.