Data Integrity Prerequisites and Other Considerations

You probably should lock your DBMS first, to avoid data corruption. If certain files are moved around while users are writing to the MYSQL or Oracle or some other database… well let’s just say it’s possible that the DBMS could get confused. The script will attempt to do this if you are using MYSQL, but you will need your mysql account information. Remember to back things up in a way you can restore them. A backup is only as good as one’s ability to restore it. Remember to store this script and all your credentials in a safe place. (chmod 700 is probably also a good idea). This is intended for backing up small instances, but you might use EBS for persistant storage, take EBS snapshots, and make copies if your instance is larger than 10 GB.

Account Credentials Needed

You might need to access your Account credentials page via aws.amazon.com. You’ll need the pem keys and the X.509 certs, etc.

Once you have all that, just edit and put this script into your cron.daily or schedule it for a good time [:

#!/usr/bin/python
# Asher Bond 2010
# Backup your EC2 instances to S3 every day
# an enhanced script based on backup.py from Paul Kenjora of Aware Labs
# http://blog.awarelabs.com/2009/painless-amazon-ec2-backup/

# apt-get install python-mysqldb if you don't have this, but
# it's not needed if you don't have MYSQL or a DBMS.
import MySQLdb

# you will need this to make system calls
import os

from datetime import date

# get this from aws.amazon.com (under security credentials you can find real credentials to replace the examples)
pem_file = '/root/.ssh/pk-APKAI6XG4IA1RVFIDNDA.pem'
cert_file = '/root/.ssh/cert-1J7DNJST6ETMFG3FI72LDODBLOS26UMS.pem'
user_id = '163265130428'
platform = 'i386'
bucket = 'backups.asherbond.com'

access_key = 'AKIAJ4QMQYJ5MXUTA4GA'
secret_key = '1WQjBA8i45CW3mnzzB/59jAFTuPl32d6u1YDgHvF'
ec2_path = '/etc/ec2/bin/' #use trailing slash

# it's a good idea to lock your DBMS so that people aren't writing to your DBMS files while they snapshot is taken.
# again, you can comment this whole database section out if you don't have a DBMS on the instance you're imaging.
conn = MySQLdb.connect (host = "localhost",
                        user = "backup",
                        passwd = "ordie",
                        db = "mysql")

cursor = conn.cursor ()
cursor.execute ("FLUSH TABLES WITH READ LOCK")
cursor.close ()

# 3 steps to back up the files to S3
days = ('monday', 'tuesday', 'wednesday', 'thursday', 'friday', 'saturday', 'sunday')
manifest = days[date.today().weekday()]

# remove some files
step_1 = ‘nice rm -f /mnt/%s*’ % (manifest,)

# bundle whatever’s here into an image
step_2 = ‘nice %sec2-bundle-vol -p %s -d /mnt -k %s -c %s -u %s -r %s’ % (ec2_path, manifest, pem_file, cert_file, user_id, platform)

# unlock the DBMS
cursor = conn.cursor ()
cursor.execute (“UNLOCK TABLES”)
cursor.close ()

# close this script’s connection to the dbms
conn.close ()

# upload the bundle to s3
step_3 = ‘nice %sec2-upload-bundle -b %s -m /mnt/%s.manifest.xml -a %s -s %s’ % (ec2_path, bucket, manifest, access_key, secret_key)

print step_1
os.system(step_1)
print step_2
os.system(step_2)
print step_3
os.system(step_3)

You should expect the process to take about 30 min on an m1.small, but t1.mini or large instances may back up faster (unless of course they have more stuff on disk, lol.)

Here is what my output looks like:

asherbond:~$ /etc/cron.daily/backup.py
Copying / into the image file /mnt/thursday...
Excluding:
/sys
/proc/bus/usb
/proc
/dev/pts
/dev
/media
/mnt
/proc
/sys
/mnt/thursday
/mnt/img-mnt
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.001796 s, 584 MB/s
mke2fs 1.41.3 (12-Oct-2008)
cBundling image file...
Splitting /mnt/thursday.tar.gz.enc...
Created thursday.part.000
Created thursday.part.001
Created thursday.part.002
Created thursday.part.003
Created thursday.part.004
.... etc ....
Created thursday.part.150
Generating digests for each part...
Digests generated.
Creating bundle manifest...
ec2-bundle-vol complete.
Uploading bundled image parts to the S3 bucket backups.asherbond.com ...
Uploaded thursday.part.000
Uploaded thursday.part.001
Uploaded thursday.part.002
Uploaded thursday.part.003
.... etc ....
Uploaded thursday.part.150
Uploading manifest ...
Uploaded manifest.
Bundle upload completed.

Be sure to log into your S3 bit bucket to make sure the backups are there. You should probably also restore each backup to a test instance and see if things look right.

Although PHP is considered to be a scripting language, PHP’s parser does in fact compile code before it is processed by the Zend Engine. Opcode caching allows compiled PHP code to be shared and recycled in memory, rather than compiling the PHP source on the fly. Opcode caching is already available in Zend Server’s community edition and will be available when PHP6 is released.

HipHop takes a much more aggressive approach to performance optimization by translating the source code into C++ which can be compiled into more machine specific instructions. Not only does HipHop compile the source code into a more system friendly language, it also rewrites your code to be more efficient. Through static binding, scripty system calls and functions such as eval() are eliminated and loosely defined types are tightened up to meet specific runtime requirements. Here is an illustration of how HipHop optimizes PHP:

According to Facebook’s Senior Engineer Haiping Zhao, HipHop reduced Facebook’s CPU overhead by 50%!

HipHop will be released tonight under the open PHP license.

1. USB
2. Firewire
3. Ethernet
4. A Camera
5. Multitasking
6. Adobe Flash Support
7. A more powerful OS than the one running on iPhones

Realistically, I don’t think keyboards are obsolete. I was hoping the iPad would be a less expensive alternative to Jazzmutant’s Lemur or Dexter, but… it’s just not happening without the proper inputs and outputs and it’s just not happening without a proper OS.

Trending Tweeters prefer iTampon instead of iPad

There is a lot of hype forming around the iPad, but iWonder how it’s better than a regular hackintosh tablet PC. iThink things will probably improve by the second generation, but iDoubt Apple will be the only one in the market by then.

For now, it seems to make much more sense to get a more powerful netbook. The iPad is supposed to cost $499-$699, but Acer offers a new Windows 7 netbook for only $299.

If you are particular about screen resolution, You can buy this 12.1 inch 1280×800 Netbook for $399

Its price is still $100 lower than the cheapest iPad… which only does 1024×768.

Here’s how targeted merchandizing can be set up in Magento Enterprise 1.6

1. Set up rules for Up-selling: For example, present the buyer with items of the same category/manufacturer as the product last viewed if the price is 10% higher.
2. Set up rules for Related Items by Subcategory: For example, automatically scan the customer’s cart and offer them items within a subcategory of the items they are planning to purchase if these items are from the same manufacturer.
3. “Customer’s who purchased product A also bought products XYZ…” – This can be done, for example, by setting a rule to show the most popular 3 products that are bought over 40% of the time when other customers bought product A.
4. These rules are followed (calculated) automatically by Magento based on the items in the customer’s cart (prior to checkout.)
5. Sometimes you know for sure when and what to up-sell: You can override these automated rules by manually selecting products that you always want to use as cross-sell, up-sell or related-items for a specific product.
6. Create customized buying options on the shopping cart page.

Targeted merchandizing is an effective way to increase sales to existing customers, but take a look at what you can do to reach new customers with targeted marketing.

Here’s how you can target new markets in Magento Enterprise 1.6

1. Set up your advertising campaign as usual with custom graphics, banners, etc.
2. Integrate banners to match your advertising campaign’s theme. Magento Enterprise offers a special smart page block that will serve the banners based on the rules created for each promotion. It could even be customized to serve as your landing page or micro-site.
3. When customers funnel in through these custom pages, you can associate promotional banners to catalog and shopping cart price rules. For example, you can offer promotional discounts without requiring users to type in promotional codes. This is done by creating call-to-action callouts in Magento Enterprise 1.6.

eBridge’s Bridgeconnect module integrates Magento Enterprise 1.6 with most Accounting and ERP software platforms including

• SAP
• Intuit
• Sage
• Epicor

You can choose to Synchronize information between Magento and your Accounting/ERP software, automatically, manually or on a set schedule. You can Bilaterally synchronize time-sensitive information such as sales orders, product information, order status and shipping information.

Magento Enterprise 1.6 brings you the Content Management features you have been asking for with CMS+

• Version Control:Store managers can create multiple versions of a static page before publishing.
• Administrators can restrict publishing privileges to only authorized users or approve pages before publishing them.
• Store Managers don’t have to know HTML. Magento Enterprise 1.6 comes with a WYSIWYG editor, which supports inserting images, URLs, product images/URLs, category URLs, and embedding videos from sites like YouTube.
• Magento Widgets: Widgets are movable, modular, configurable (and pre-configured) page blocks that allow you to display dynamic (or static) content on your store front-end. For example display all best sellers for a specific category on the top part of the category page. This is something Magento users have hand-coded until now.

These are only a few of the new features available in Magento Enterprise 1.6. For more information, visit Magentocommerce.com.

The ajax behind the scenes is a lot more streamlined as well because apps and other notifications don’t have to load in the background every time you click something.

Help!! I can’t get my Applications! How do I turn off Facebook Lite and go back to the normal Facebook?”

Don’t panic. Just type “http://facebook.com” into your address bar and you will go back to your normal Facebook with your Mafia Wars, FarmVille, Friends for Sale, and other useless applications. To get back to lite mode, type the URL “http://lite.facebook.com” into your address bar again.

Fatal error: Call to a member function children() on a non-object …

The most common reason for this error message is that the Magento Installation script is trying to access other scripts which are in a directory that is unreadable by your web server. You probably broke the file and directory permissions for these installation scripts.

You might have better luck with uncompressing the tar.gz archived version of Magento, which you can download from Magentocommerce.com. Tar archives generally are preset with the permissions of the person who archived them, which might be easier since the good folks at Magento probably have a better idea about how permissions need to be set for their software.

If you are using a Magento friendly hosting company like Host Monster, you should be able to simply upload the compressed tar.gz file and uncompress it on the server side with the permissions already set properly.

It’s still a good idea for you to know how to set permissions for web applications on a web server, especially if you run your own VPS or dedicated web server or people are paying you to install/set up Magento for them.

Here’s how to set the file and directory permissions for Magento Manually

Check that all directories inside your magento directory are readable and executable by your web server’s user. Normally the web server runs as either the user apache or nobody. As a side note, it’s more secure to run your web server as it’s own username instead of “nobody”.

For those of you running the lazy (less secure) installation of your web server… The lazy (less secure) way is to change permissions is:
chmod -r 777 /your/magento

Don’t set permissions using the lazy method, because anyone with access to your web server (shell, ftp, or even just web access through a script) can overwrite your magento files and heavily compromise your e-commerce site. It is very important to run an e-commerce web server securely because a compromised web server can send thousands of credit card numbers to thieves before the security breach is noticed. If you are unsure about security, it is recommended that you use a scalable e-commerce service such as Shopify or that you purchase a Magento installation from someone who is seasoned in e-commerce. I can set up the Magento software platform (which supports the hosting of multiple web stores) in about an hour or two depending on your hosting situation. However, if you prefer to and are comfortable setting up e-commerce sites yourself, read on.

Try to limit permissions to only your ftp/shell username and the web server. The way most people handle this is to change the owner of files to their ftp/shell username and set the group to the web server’s group.

For example (assuming your web server runs under the user name “apache”):
chown -r yourname:apache /your/magento

Make files (not directories) readable by your web server:
find /your/magento -type f -exec chmod 640 {} \;

The next thing you should do is set the file permissions for Magento directories to readable and executable:
find /your/magento -type d -exec chmod 750 {} \;

Now set the permissions for directories that need to be writable and executable:

chmod 770 /your/magento/app/etc
chmod 770 /your/magento/var
chmod 770 /your/magento/media
chmod 770 /your/magento/media/downloadable

chmod 770 /your/magento/media/import

United States Postal Service:
This shipping method is currently unavailable. If you would like to ship using this shipping method, please contact us.

This is because you aren’t getting proper access to the USPS API. Magento, by default comes with the following URL set up in the shipping configuration for USPS:
http://production.shippingapis.com/ShippingAPI.dll

The problem is that by default, you don’t have access to USPS’s production API. They have to approve you first. Keep in mind that they only allow you to use it on one web site per account, so if you’re setting Magento up for a client, customer, or other business entity other than your own you should set up a separate USPS account for them.

You can set up your USPS WebTools account here:
http://www.usps.com/webtools/

Once you have set up your account, they will give you access to the testing environment. The URL for the USPS test API is:
http://testing.shippingapis.com/ShippingAPITest.dll

Now, before you paste that into Magento, you should be aware that the USPS test environment only supports very specific data. According to their own documentation:

ZipOrigination: 10022
ZipDestination: 20008 (is set during Checkout process by user)
Pounds: 10
Ounces: 5 (weight can be set for every product on Administration > Products)
Size: LARGE
Machinable: TRUE

To test international shipping use the following values:

Pounds: 2
Ounces: 0
MailType: Package
Country: Albania

or

Pounds: 0
Ounces: 1
MailType: Postcards or Aerogrammes
Country: Algeria

All other parameters’ values won’t work in the test environment.

This is why most Magento consultants and installation service providers typically just ask USPS to turn on production mode. When you are approved by USPS for the production API, you should use the following URL:
http://production.shippingapis.com/ShippingAPI.dll

But remember, it’s very important that you set up a separate USPS account for each Magento web site (or at least one per domain). USPS will disable your access to their production server if you are caught using your account on multiple web sites. That would make all of your Magento customers very upset, all at the same time.

To request access to the USPS production API, fill out the form here:
http://www.usps.com/webtools/webtoolsapirequestform.htm

Be sure that in your request, you explain that you are using third party software (Magento) and that you only want access to the API so that you can calculate shipping costs associated with using USPS.

If you leave out this information they will reject your request or ask you to submit a valid test before they give you access. Third party software doesn’t need to be tested as long as you tell them you are using Magento.

USPS doesn’t allow you to use their API for batch processing or data cleansing, so be sure to NOT check these boxes on the request form.

Once USPS gives you access to their production API server, Magento should work with all USPS shipping options. If you are still having trouble, check that you have access to USPS’s production API server.

USPS Tech Support: 1-800-344-7779 (7:00 AM to 11:00 PM EST daily)

Once you have it correctly configured and you have access to the production API, you should be able to place an order and check out to see a list of USPS shipping options with the costs of each calcualted based on the weight of the product you are ordering.

You will need to remove shipping options that are irrelevant, for example USPS Library Mail is only for shipping academic things like books between academic institutions or libraries.

Click here to read about different shipping methods such as Parcel Post.

Here is some PHP code you can use in wordpress to get the tags of your post ID.
[source:css]
$tags = wp_get_post_tags( $post->ID );
[/source]

Let’s assume the result is one category (264) and one tag (41).

1. If you are not using WordPress Mu, you can simply select from wp_term_relationships and wp_posts
2. If you are using WordPress Mu, you will need to specify the tables as wp_MU-BLOG-ID#_term_relationships and wp_MU-BLOG-ID#_posts. Let’s assume we are using WordPress Mu and our blog ID is 1. The tables we are working with are wp_1_term_relationships and wp_1_posts

[source:css]SELECT p.ID, COUNT(tr.object_id) AS cnt
FROM wp_1_term_relationships AS tr, wp_1_posts AS p
WHERE tr.object_id = p.id AND tr.term_taxonomy_id IN(264,41) AND p.id!=3987 AND p.post_status=’publish’
GROUP BY tr.object_id ORDER BY cnt DESC, p.post_date_gmt DESC LIMIT 5;[/source]

Notice that we have limited our results to the 5 most relevant posts. It is recommended that you limit this query because it can be a difficult query to process if you are running it on a blog that has thousands of posts. You can change LIMIT 5 to LIMIT 25 or so on blogs that are hosting less than 5,000 posts without seeing much of a difference in performance. On larger, more frequently updated blogs or WordPress MU sites which are hosting a lot of blogs on the same server, it is recommended that you cache the results of this query and only use it once an hour or once every time a blog entry has been updated.

To add these extentions, I had to recompile php adding these two parameters to my PHP 5 configure script:
–with-mcrypt
–with-pdo-mysql