1. Create a multipart/alternative email envelope.

from email.mime.multipart import MIMEMultipart
alter = MIMEMultipart('alternative')
msg = MIMEMultipart()

This will create an envelope that will contain all 3 mentioned parts. In mutt’s view:

1                     [multipa/alternativ, 7bit, 16K] 
2 ├─>                 [text/plain, base64, utf-8, 0.1K] 
3 ├─>                 [text/html, base64, utf-8, 0.1K]  
4 ca.crt                              [applica/octet-stre, base64, 1.5K]

Some MIME type is truncated, but you get the idea.

2. Unicode
You will have to append the coding part in the source file.

#!/usr/bin/python
# -*- coding: utf-8 -*-

The second line must exist.

Then a unicode title and text:

from email.mime.text import MIMEText

# msg and alter is derived from the previous example
msg['Subject'] = u'中文'
text = u'正在用Python写东西'
alter.attach(MIMEText(text, 'plain', 'utf-8'))

msg.attach(alter)

Appending HTML just change the plain to html when you call MIMEText against HTML content.

3. Add attachments
Mind the relative / absolute path from the location you execute the script. It’s recommend to use absolute path.

from os.path import basename, dirname, realpath

# assuming I will attach all the files in the same location as the script itself, also a file from other relative path.
cwd = dirname(realpath(__file__))
files = ['myfile.bin', 'herfile.bin', '../other/dir/s.bin']

for f in files:
        # use absolute path
        with open(cwd + '/' + f, "rb") as fil:
            msg.attach(MIMEApplication(
                fil.read(),
                Content_Disposition='attachment; filename="%s"' % basename(f),
                Name=basename(f)
            ))

4. Message ID.
Some SMTP server supports adding message id to your email on a specific port, e.g. 25. But when you use starttls to connect the SMTP or use a different port, e.g. 587, Message-Id may not be added automatically. So your email probably won’t be received as it’s required in RFC 2822.

One way to get around that is to generate your own message id. Then you could use starttls() to send your emails securely.

from email.utils import make_msgid

# Inherit the msg from above example.
msg['Message-Id'] = make_msgid('example.com')

smtp = smtplib.SMTP('smtp.example.com', 587)
smtp.starttls()
smtp.login('sender@example.com', 'emailpassword')
smtp.sendmail(sender, to, msg.as_string().encode('utf-8'))
smtp.quit()

Note that the message id it generated will use the host name of your current machine if you do not specify.

...
verbosity=INFO
...

On VM2 it is set to ERROR, VM3 is set to DEBUG, etc.

TL;DR: https://github.com/hlx98007/deployment-scripts/blob/master/chef_scripts/merge.py

We know this can be separated by Chef Environment or Chef Role depend on how you set it up. Let’s say, I am using environment for this configuration.

{
  "name" : "vm1"
  ...
  "log_level" : "INFO",
  ...
}

I assume that you can derive the other VM template file on your head.

OK. I happen to manage a large group of Chef managed laboratories during my internship. We have several development environments and several test environments. And those values are apparently not secret (to our employees). So we have the need to manage them. With the complexity of our product, if any new key is added to, or some old keys are deleted from the environment file, we need to add/subtract the same key to each environment file across all my responsible labs.

This is a pain!

So we have a brilliant idea to include all labs detail into 1 master template.json. Like this:

{
  "name" : { // We must have a "name" key due to it's Chef!
    "default" : "",
    "vm1" : "vm1",
    "vm2" : "vm2",
    ...
  },
  ...
  "log_level" : {
    "default" : "INFO",
    "vm2" : "ERROR",
    "vm3" : "DEBUG",
    ...
  },
  ...
}

Looks neat? The keyword to any merged template is the “default” keyword, or anything you define. As long as we just keep the 1 template file, we can version control it, tag it however we like. In the Python script I wrote, I used “template_default” keyword.

We will talk about how to extract those values in the next blog post, here, we will talk about how to merge those files in Python.

In Python’s eye, JSONs are dictionaries. Dictionaries can have indefinite depth and it is not easy to create the same structure with an empty dict. I believe it is best to do this recursively, calling a merge() to a smaller subset of itself, and create the result to a new dictionary.

Each time when we call the merge(), we will create an empty dictionary to hold the copied values and eventually return to the upper function. Eventually we will have the correct structure and data.

Consider the following scenarios and my decisions when we call merge.py env1.json env2.json template.json, we will extract the keys on the current level only for env1.

for key in env1_json.keys()

If env2 has the new key, we can do merge.py env2.json template.json template2.json to merge the missing keys. (notes at the bottom)

0. value of env1 is exactly the same to env2. (copy and continue)
1. value of env1 is a string, value of env2 is a string
  1.1 they equal (copy and continue)
  1.2 they differ (build the template structure)
2. value of env1 is a string, value of env2 is a dict
  2.1 env2 is not a template style dict (notify and stop)
  2.2 env2 is a template style dict
    2.2.1 env1 does not exist (add)
    2.2.2 env1 does exist but with different value (update)
    2.2.3 env1 does exist and with the same value (update)
3. both values of env1 and env2 are dicts.
  3.1 env1 is a template style dict (stop and report)
  3.2 env2 is a template style dict (stop and report)
  3.3 env1 and env2 are both normal dicts (recursive call on itself)

With those points sorted out, the program is not difficult to write.

Known features/bugs that not yet categorized:
1. When env1 has some missing keys and merge with a template json, such keys will be lost in merged.json. (may be we do want to delete that key in the template?)