Jurgens du Toit

2017-02-02T05:34:34+00:00

I like frameworks. I like working with them, I like writing them. I wrote Backend-PHP, which I’m using personally and at Branded Internet. Unfortunately it has a number of major flaws: It’s not easily unit testable, not really a REST web service, and it doesn’t conform to any popular coding standards. So I wrote Backend-Core as the first step towards addressing these issues.

Minimal

Backend-Core is only intended as a minimal, core system. It doesn’t implement any application logic or data source abstraction. I decided to focus on the core functionality and design of a RESTful MVC system, and make sure that it works as it should. Eventually I’ll extend it with a seperate code base that will include application and data source logic.

MVC

The system is structured into Model, View and Controller components:

Model - The model contains all the business logic for an application. As this is a core system, the models are empty.
View - The view determines how the results of a query will be displayed.
Controller - The controller contains the application logic. Once again, this is only the core, not an application, so the controller just calls the model.

I found that a number of MVC systems confuse what should go into the Model and what should go into the Controller. I’m aiming for thin Controllers containing application logic, and thick Models containing the business logic.

Unit Testing

Backend-PHP used static global classes quite liberally, without even a thought around dependency injection. This makes unit testing unreliable and a real headache to add to the framework.

Backend-Core, on the other hand, was designed to use constructor injection across the board, and includes a number of unit tests, supported by PHPUnit. This should increase the overall stability of the system.

RESTful Service

Something I didn’t understand about REST when I initial wrote Backend-PHP, was that every REST URL should refer to a noun, with the only verbs being GET, POST, PUT and DELETE, as specified by the HTTP request. This resulted in it not complying with REST standards, as a lot of the URLs contained extra verbs, or didn’t refer to nouns directly.

Backend-Core supports the REST / HTTP verbs directly, and uses the same methods as CakePHP to identify which verb to use:

A _method POST variable
A X_HTTP_METHOD_OVERRIDE header sent with the request
The HTTP request’s method

This enables it to be used from a Web browser as well as from a REST compliant client.

Documentation and Coding Standards

The code complies with the ZEND coding standard, as enforced by PHP Code Sniffer. It’s documented using the PHPdoc standard. Hopefully this will make it easier for people to contribute to and extend the system.

Feel free to fork, comment and use this.

The Logstash Config Guide

2015-01-12T00:00:00+00:00

The ELK stack has been becoming more and more popular, specifically Elasticsearch. The search volume for it has been growing constantly since 2010. It’s faithful sidekick, Logstash, has been growing along with it, and ever since I’ve started looking at it, I’m just loving it more and more.

The fact that you can string together totally disparate systems, from XMPP to Redmine and files on S3 to Nagios, with a simple config makes it a wonderful plaything and very useful utility. The rub, though, is in the config: Even though a great effort is made to write proper docs, I’ve found the documentation to be unclear, incomplete and sometimes confusing. So I’ve decided to do something about it:

Pain Free Logstash Configuration

Buy it now on Leanpub

The plan is to write focused guides on some of the more popular Logstash plugins in the hopes of making it easier for people to set up and configure them. Each chapter will point out the bare minimum required to get you up and running and also highlight some of the more interesting, but not necessarily necessary settings for a plugin. You’ll also find a complete reference of all the available settings, with working examples for each of them.

It’s still early days, but I’ll be publishing a number of preliminary posts on [EagerELK][5] to get things going. Subscribe there, or just buy the book on Leanpub, to stay up to date.

The book will be focusing on version 1.4.2 of Logstash (since 1.5 is still in beta), but I’ll update it once 1.5 is stable and being adopted. If you sign up for the 1.4.2 version, you’ll receive a free upgrade to the 1.5 version. Two books for the price of one!

Any questions or comments are welcome, let me know below!

Populate a Symfony 2 Form with the referring entity

2014-02-03T00:00:00+00:00

In any web project it often happens that you have an entity, say a Group, to which you want to add a linked entity, say a Student. So you’ll have a “Add a Student” link on the page displaying the Group. When your users click through, there’s probably a dropdown with the different groups, and your users expect the Group they’re coming from to be prepopulated.

There’s various ways to do this, but I’d like to show you a simple, non intrusive one for Symfony 2. You don’t have to add any parameters to the link, it just works. In your Student Controller, the newAction method:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24 <?php
// src/My/Bundle/Controller/StudentController.php
    public function newAction()
    {
        $entity = new Student();
        $form   = $this->createForm(new StudentType(), $entity);

        // Get the refering URL Path
        $ref = str_replace("app_dev.php/", "", parse_url($request->headers->get('referer'),PHP_URL_PATH ));
        // Get the matching route
        $route = $this->container->get('router')->match($ref);
        if (empty($route['_route']) === false && $route['_route'] === 'group_show') {
            // Find the referring group
            if ($group = $this->get('doctrine')->getRepository('MyBundle:Group')->findOneById($route['id'])) {
                $form->get('group')->setData($group);
            }
        }

        return $this->render('MyBundle:Student:new.html.twig', array(
            'entity' => $entity,
            'form'   => $form->createView(),
        ));
    }
?>

We basically try and match the referring URL to a route, and if found, retrieve that entity, and set it in the form. Simple!

MS-SQL Stored Procedures in Sequel: Getting the value of output variables

2014-01-03T00:00:00+00:00

We had the opportunity to muck around with a couple of technologies at Tutuka the last few weeks. I managed to do some work with Ruby, specifically Sinatra and Sequel. We use MS-SQL extensively and eventually we ran into an issue where we couldn’t get the values of output variables from stored procedures. After extensive googling we found out that it’s not supported. There were a couple of clues on how it might be done (particularly this TinyTDS Issue), so after chatting with Jeremy (the maintainer of Sequel) I decided to try and the necessary support.

It Worked!

I was relatively surprised on how easy it was. As of yesterday (2014-01-02) it’s now part of the Sequel version 4.6

Add Database#call_mssql_sproc on MSSQL for calling stored procedures and handling output parameters (jrgns, jeremyevans) (#748)

How to use it

You can get a good idea on how to use it from the documentation, but here’s a summary.

Let’s say your stored procedure is defined as follows:

CREATE PROCEDURE dbo.SequelTest(
  @Input varchar(25),
  @Output int OUTPUT
)
AS
  SET @Output = LEN(@Input)
  RETURN 0

If you don’t care about the type or the name of the output variable, execution is as simple specifying that an argument is an output parameter by passing the :output symbol as the argument value.

DB.call_mssql_sproc(:SequelTest, {:args => ['Input String', :output]})

> {:result => 0, :numrows => 1, :var1 => "1"}

The result and numrows element will contain the result code returned by the stored proc and the number of rows affected respectively.

If you need to specify the type of the output variable, do so by specifying the second element of the array:

DB.call_mssql_sproc(:SequelTest, {:args => ['Input String', [:output, 'int']]})

> {:result => 0, :numrows => 1, :var1 => 1}

If you need to specify the name of the output variable, do so by specifying the third element of the array:

DB.call_mssql_sproc(:SequelTest, {:args => ['Input String', [:output, nil, 'output']]})

> {:result => 0, :numrows => 1, :output => "1"}

Enjoy!

PHP: The slums of the Programming World?

2013-10-10T00:00:00+00:00

I presented this at Tech4Africa 2013. The slides can be found here.

Quick History of PHP

PHP was initially created by Rasmus Lerdorf when he got irritated by repetitive coding of CGI calls. He consolidated a number of functions into a C library and created a parser which would call these functions out of HTML. And so PHP was born! Versions 2 and 3 were formalizations and extensions of the libraries, still using Rasmus’ parser.

The Zend Engine replaced Rasmus’ parser for version 4, introducing OOP capabilities, and in Version 5 Zend Engine 2 was introduced. Recent versions of PHP saw even more improvements, most notable namespaces in 5.3 and a built in web server and traits in 5.4

There’s no ideology or design philosophy, except that it should be a simple tool to solve problems.

“I’ve never thought of PHP as more than a simple tool to solve problems”

Rasmus Lerdorf 2010

Lies, Damned Lies and …

Despite a worsening reputation, the language hasn’t decreased in popularity.

Less searches are being done for PHP.

If you look at the JavaScript vs JQuery trends, JavaScript is also decreasing, but JQuery is increasing. It seems like the programming community is realising that frameworks are a good idea.

So if you look at PHP frameworks, searches on them are also increasing, especially after 2013 when Zend 2 and Symfony 2 were released.

No shortage of hosts offering PHP.

Who uses it?

User facing portion of Facebook through HipHop PHP (which generates C++)
Wordpress - Application turned framework
Yahoo (for now). They are replacing it with Node.
Wikipedia / Mediawiki

Embrace the Cactus

It’s easy to create insecure / buggy code in PHP:

    mysql_query("SELECT * FROM `users` WHERE `username` = $_GET['username']");

PHP accounts for only 18% of applications covered by the CVE, but accounts for almost 29% of the vulnerabilities

It’s developed a bad reputation in the programming community:

Bringing a knife to a gun fight is like bringing PHP to a modern web stack.

@iamdevloper

Settings such as register_globals and magic_quotes that defaulted to on opened up a lot of security holes

No unified community. In general, the community is great. In the rare cases where the PHP documentation (or even the language) is insufficient, you can usually find something of value in the comments. Yet, there is a lot of conflict in both the official (like PHP internals) and the unofficial (like PHP-FIG) communities.

If you look at the debate on what to use for indentation, tabs vs spaces, the community is in total disarray.

Inconsistencies in naming functions:

strrev     // String Function
str_repeat // String Function
explode    // String Function
// At least somewhat consistent
strpos($haystack, $needle)
array_key_exists($needle, $haystack)

Object Oriented, or just Object Capable? - Michael Kimsal

So you can do Object Oriented Programming as well as Procedural and Functional Programming in PHP (especially PHP 5.3 and later).

See you on the flip side…

PHP 5.3

PHP 5.3 is for me a watershed version on PHP’s history - It introduced a number of features / capabilities that made it possible for some of the other factors I’ll mention to come into existence. The most notable feature is namespaces, making it easier to name classes in a sane way. A lot of people welcomed the introduction of lambda functions and closures.

I think the PHP developers and community realized that PHP was becoming outdated and had a lot of valid issues that needed to be addressed. For me 5.3 was a definite step in the right direction. It introduced (amongst other things):

Namespaces
Closures and Lambda functions

PHP-FIG

Although it seems to generate a lot of conflict, the PHP-FIG has managed to at least get the different frameworks talking. It’s an unofficial (as in not sponsored by PHP) group formed by various frameworks and projects whose aim it is to get the projects talking and to find ways to work together. It’s not always plain sailing, but it seems to get to valuable decisions. Eventually.

PSR-0 has enabled the creation of packaging software such as composer / packagist
PSR-1 and PSR-2 generated a lot of conflict, but at least all of the participating frameworks partially agreed on coding standards
PSR-3 and PSR-4 aren’t momentous decisions, but they at least pave the way forward for increased collaboration

Packagist / Composer

Composer is a tool to manage dependencies for a project, and manages and installs dependencies on a per project basis. Very much on par with node’s npm and ruby’s bundler. Packagist is a repository for composer packages and makes it dead simple to find a package / library that will address your specific need. Before this, we had PEAR, but that was usually a headache to set up and version properly locally, not to mention on a shared host. Now it’s as simple as doing

    composer require "vendor/library"

To a certain extent the improvements in PHP 5.3 and some of the work from PHP-FIG made this possible, although composer could have existed without them.

Modern Frameworks

The combination of PHP 5.3 (specifically namespaces and proper OO support), the work of the PHP-GIH and the development of Composer made it possible to not only create frameworks such as Symfony2, Zend2 and Laravel, but also to reuse components from these frameworks in other projects. Drupal 8 uses a lot of Symfony2’s components. PPI is a framework built using both Zend2 and Symfony2 components. You can now easily choose and combine libraries without having to manage class names or dependencies. This has at least two direct advantages for project developers:

Instant access to a whole range of precoded, quality, libraries.
No more wasting time on managing dependencies and name clashes.

So, Slum or Diamond in the Rough?

Good systems cater for users’ mistakes, great systems also cater for developers’ mistakes…

Jrgns

My opinion is, there is no right answer. To a certain extent, PHP can be described as a horrible language that makes it very easy to introduce bugs and security issues. It doesn’t give you guidance on how to do anything, and can’t seem to decide if it’s procedural or object oriented. In the same breath, it gives you a lot of freedom to code according to your style, and makes a lot of common coding tasks very easy to do. Successful business have been built and are still running on PHP.

If you’ve never coded anything but PHP and have no formal programming education (including good programming books), you’re probably coding crappy PHP. Probably. I’d suggest you start reading up on good programming practices, start reading code that’s of a high quality, and start learning other programming languages, especially stricter ones like Java, or start using frameworks such as Symfony2 and Laravel that at least guides you in the right direction. Speaking multiple natural languages keeps your brain sharp and young. Coding in multiple languages makes you a better coder.

If you know what you’re doing and you perhaps have some extra experience in a more structured language such as Java or Ruby, you’re probably coding quality PHP. It’s un-opinionated, so, as long as you know what you’re doing, it can be a great tool. Don’t discount the language just because it’s lost it’s coolness factor. You can write quality code in PHP.

If you can pull it off, you’re badass as hell, but most of the time you’re only going to end up stabbing yourself.

ohpalmer

A DIY Dynamic DNS service using Cloudflare

2013-09-26T00:00:00+00:00

DynDNS was always the dynamic IP service of choice, but I could never get it to work properly, and I was always stuck with their lame domain names as I’m too cheap to pay for their serivces.

Luckily services of all kinds, including DNS Providers, have caught on to the fact that people want API’s. If there’s an API to do DNS updates, I can script a solution.

Some of my domains run off of Cloudflare which needs to overwrite your DNS. And, hey presto, they have an API. So here it is:

1
2
3
4
5
6
7
8
9
10
11
12 #!/bin/bash
curl https://www.cloudflare.com/api_json.html \
    -d 'a=rec_edit' \
    -d 'tkn=1234512345qweqwe1234512345' \
    -d 'id=12345' \
    -d 'email=jrgns@jrgns.net' \
    -d 'z=jrgns.net' \
    -d 'type=A' \
    -d 'name=home' \
    -d "content=$1" \
    -d 'service_mode=0' \
    -d 'ttl=3600'

Quick and dirty. This updates a domain (home.jrgns.net) with id 12345 with the IP address that’s passed as the first argument to the script. There’s a bunch of ways to get your external IP, my favourite is curl http://api.externalip.net/ip, but I just run this script as the postscript of my ddclient (which updates another service)

1
2
3
4
5
6
7
8
9
10 # /etc/ddclient.conf

protocol=dyndns2
use=web, web=myip.dnsomatic.com
ssl=yes
server=updates.someservice.com
login=mylogin@gmail.com
password='123456'
daemon=3600
postscript=/home/jrgns/bin/cloudflare_dynamic_dns.sh

I’m sure the same can be done with outer services such as Route 53? Has anyone tried something like this?

Setting a default order for Doctrine 2 / Symfony 2

2013-09-17T00:00:00+00:00

Usually when you’re working with data you expect some kind of default ordering. It might be the primary key, or, more commonly, the data set will be ordered by a timestamp field or the data’s most important property. In Doctrine you can create custom repository classes to tweak queries to your liking:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18 <?php
// src/Acme/StoreBundle/Entity/ProductRepository.php
namespace Acme\StoreBundle\Entity;

use Doctrine\ORM\EntityRepository;

class ProductRepository extends EntityRepository
{
    public function findAllOrderedByName()
    {
        return $this->getEntityManager()
            ->createQuery(
                'SELECT p FROM AcmeStoreBundle:Product p ORDER BY p.name ASC'
            )
            ->getResult();
    }
}
?>

This is all fine and dandy, but do you really want to repeat yourself everytime you write a query for a model that will almost always be ordered by the same field? Or get caught out by calls to default repository methods that don’t use the ordering you like? No, you don’t. Enter this piece of code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21 <?php
// src/Acme/StoreBundle/Entity/ProductRepository.php
namespace Acme\StoreBundle\Entity;

use Doctrine\ORM\EntityRepository;

class ProductRepository extends EntityRepository
{
    public function findBy(array $criteria, array $orderBy = null, $limit = null, $offset = null)
    {
        // Default Ordering
        $orderBy = $orderBy === null ? array('added' => 'desc') : $orderBy;

        // Default Filtering
        if (array_key_exists('active', $criteria) === false) {
            $criteria['active'] = 1;
        }
        return parent::findBy($criteria, $orderBy, $limit, $offset);
    }
}
?>

The findBy method of the EntityRepository class is the method used by most (if not all) of the the repository methods that fetch data. This includes findBy methods where you can specify an arbitrary column and value:

1
2
3
4 <?php
// find a group of products based on an arbitrary column value
$products = $repository->findByPrice(19.99);
?>

So to ensure a default ordering, we override this method and check if an orderBy clause has been set. If not, set it to our default ordering — in this case by the added field, descending. For good measure we also add default filtering — the active field needs to equal 1. Once that’s done, call the parent method with the default values set.

Easy as π

Pre-populating an Entity in a Symfony 2 Form

2013-07-22T00:00:00+00:00

I’ve been working on teach-a-rific.com in my spare time, and I’m learning a lot about how Symfony 2 Forms work. One thing I need to do a lot, is to show a form with a entity field pre-populated. In this situation you also don’t want the user to be able to edit the field. This is how I did it.

First, you need to retrieve the mapped entity, in this case a Student, and set it in the entity which you’re creating a form for. In this case a ContactDetail. This all happens in the newAction method of the ContactDetailController. Notice how I save the student ID in a session variable. More on this later.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17 <?php
//ContactDetailController::newAction
$detail = new ContactDetail();

if ($studentId = $request->query->get('student_id')) {
    $student = $this
        ->getDoctrine()
        ->getManager()
        ->getRepository('NotesNoteBundle:Student')
        ->find($studentId);
    if ($student) {
        $this->get('session')->set('contact_detail:create:student', $studentId);
        $detail->setStudent($student);
    }
}

$form = $this->createForm(new ContactDetailType(), $detail);

On to the form. Quite simply, if the entity already has a Student, don’t add the field for the student.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18 <?php
use Symfony\Component\Form\AbstractType;
use Symfony\Component\Form\FormBuilderInterface;
use Symfony\Component\OptionsResolver\OptionsResolverInterface;

class ContactDetailType extends AbstractType
{
    public function buildForm(FormBuilderInterface $builder, array $options)
    {
        if ($builder->getData()->getStudent() === null) {
            $builder->add('student');
        }
        $builder
            ->add('type', 'choice', array('choices' => array('Email' => 'Email', 'Cellphone' => 'Cellphone')))
            ->add('content')
        ;
    }
}

On to the actual form. If the student field is defined in the form, display the form widget. Otherwise display the student in an uneditable text input. Displaying the student can be skipped if it’s not necessary.

1
2
3
4
5
6
7
8
9
10 {% if form.student is defined %}
    {{ form_row(form.student, { form_type: 'horizontal' }) }}
{% elseif form.vars.data.student.id is defined %}
    <div class="control-group">
        <label class="control-label">Student</label>
        <div class="controls">
            <span class="input uneditable-input">{{ form.vars.data.student }}</span>
        </div>
    </div>
{% endif %}

And on to the updateAction. Just get the Student whose ID we stored in the session, and set it on the ContactDetail before populating the form:

1
2
3
4
5
6
7
8
9
10
11
12
13
14 <?php
//ContactDetailController::createAction
$entity  = new ContactDetail();

if ($studentId = $this->get('session')->get('contact_detail:create:student')) {
    $student = $this
        ->getDoctrine()
        ->getManager()
        ->getRepository('NotesNoteBundle:Student')
        ->find($studentId);
    if ($student) {
        $entity->setStudent($student);
    }
}

Simple enough. Any suggestions on how to improve it, or comments on how to do it differently?

A Summary of Symfony 2 Authorization

2013-04-29T00:00:00+00:00

Symfony 2 comes with a great security component that provides both Authentication (identifying the user) and Authorization (determining the permissions for the user) functionality. It has sane defaults, works out of the box, and is extendable enough to cater for any scenario. This post describe the default authorization functionality available to the developer.

Out of the box, Symfony 2 offers two method with which to authorize a user:

Role based authorization
Access control lists (ACL’s)

Role Based Authorization

With role based authorization, you specify a number of roles (which can be hierarchical) and specify the access of these roles, all within your application config:

#project/app/config/security.yml
security:
    role_hierarchy:
        ROLE_ANONYMOUS:
        ROLE_USER:        ROLE_ANONYMOUS
        ROLE_ADMIN:       ROLE_USER
        ROLE_SUPER_ADMIN: [ROLE_ADMIN, ROLE_ALLOWED_TO_SWITCH]

    access_control:
        - { path: ^/login, roles: IS_AUTHENTICATED_ANONYMOUSLY }
        - { path: ^/public, roles: IS_AUTHENTICATED_ANONYMOUSLY }
        - { path: ^/, roles: ROLE_USER }
        - { path: ^/admin/, role: ROLE_ADMIN }

In the config above, there’s roles (or types of users). Anonymous, normal users, admin users and super admin users. Anonymous users can only access the public part of the site. Logged in users can access the whole site, except for the admin area. Admin users can access the admin area of the site. Relatively simple and easy to set up, but not very customizable on a fine grained level.

Access Control Lists

Say for instance that users generate content, and that users can only access the content they generated. The role based rules are too general for this, so we’ll turn to Access Control Lists for a solution.

The concept of ACL’s boils down to a list of many to many relations between users and resources defining who has access to what. Even though Symfony provides built in ACL functionality, this can become hairy quite quickly, as you need to manage Access Control Entries (ACE) in your controller code.

Although Symfony provides an ACL Provider with which the coder can create and check ACL’s, it can become hairy quite quickly, as it needs to be managed in each and every Controller Action.

This boilerplate code wil appear in controller actions that deal with creating Entities and their ACL’s:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20 <?php
use Symfony\Component\Security\Acl\Domain\ObjectIdentity;
use Symfony\Component\Security\Acl\Domain\UserSecurityIdentity;
use Symfony\Component\Security\Acl\Permission\MaskBuilder;

// Inside a controller action

// creating the ACL
$aclProvider = $this->get('security.acl.provider');
$objectIdentity = ObjectIdentity::fromDomainObject($entity);
$acl = $aclProvider->createAcl($objectIdentity);

// retrieving the security identity of the currently logged-in user
$securityContext = $this->get('security.context');
$user = $securityContext->getToken()->getUser();
$securityIdentity = UserSecurityIdentity::fromAccount($user);

// grant owner access
$acl->insertObjectAce($securityIdentity, MaskBuilder::MASK_OWNER);
$aclProvider->updateAcl($acl);

The “creating the ACL” block retrieves the Security Provider, generates an identity for the object using the ObjectIdentity class, and then creates the ACL using the security provider.

The second block retrieves the Security Context and the current user. From there it generates the an identifier for the user using the UserSecurityIdentity class.

The third block then inserts an Access Control Entry and updates the ACL.

That’s a lot of code. If there’s a generic way to handle this, please let me know in the comments.

Behind the Scenes

The key component in how Symfony manages authentication is the AccessDecisionManager. Each time a request is made, the AccessDecisionManager lets registered Voters vote on if the user may access the resource. The voters will determine their vote by inspecting the current security token, a set of attributes, and an optional object. Access will then be granted (or denied) depending on the votes and the AccessDecisionManager's decision strategy. Both the role based auth and ACL’s use this mechanism.

I’ll go into how to use the AccessDecisionManager and Voters in a later post.

I'm the most Organized Disorganized person you'll meet

2013-02-28T00:00:00+00:00

One of the most telling compliments I ever received was from a good friend: “You’re the most organized, disorganized person I know!”. It’s quite telling once you wrap your head around it. It stuck by me, and it’s helping me shape my career.

I’ve always been absent minded. Classic “if you’re head wasn’t tied to your body, you’ll loose it!” stuff. I’d put something down, and then search for it for half an hour, only to discover it’s been in my pocket the whole time. One time I was talking to my mom on the phone, realised that I didn’t know where my phone was, and proceed to tell my mom about it… At the end of the day, details just didn’t stick in my head.

Now I’m in an industry where detail is everything. Businesses lose thousands because of a missing negative sign. One line of code out of place crashes an application and loses a company users, money and its reputation. I always tell people that creating systems is like eating an elephant: bit by bit. what they don’t realise is that if you get one byte wrong, everything comes crashing down.

So how wise is it for a guy who can’t keep track of both his sandles to be in the details business? Either very foolish, or a stroke of genius. Genius? Let’s look that compliment again: The most organized disorganized person. To cope with all these details, I started to rely on systems. I organized my life in such a way that I didn’t need to worry about the details.

Using systems ensures that even the most unorganized person won’t miss any of the details. I used calandering to keep track of appointments. A todo list for tasks. Evernote for easy to forget yet important info. When I go out the door I chant “Phone, keys, wallet” to ensure that I have the essentials. Every day I’m hacking my brain.

The take away is that while people make mistakes, systems rarely do. Don’t rely on people to take care of the details, rather put systems in place to take care it. If it’s tricky or complicated to deploy an app, script it. If a process is long and protracted, automate it. Create unit and behaviour tests for your code. Add state checks to the application. Run regular audits on the numbers and make sure they balance.

In the age of rapid development and continuous deployment, just writing your app isn’t enough anymore. You need to write systems around your app to ensure that it is coded correctly, deployed correctly, working correctly.

PHP's array_merge_recursive function explained

2013-01-08T00:00:00+00:00

Reading the PHP docs, it’s not immediately clear how the array_merge_recursive function behaves. I’ve setup a couple of test cases and compared them with the plain array_merge function to explain it further:

The following snippet set’s up a couple of arrays, and outputs them for completeness sake.

1
2
3
4
5
6
7
8
9
10
11
12
13
14 <?php
$cats = array(
    'white' => array('Kitty'),
    'brown' => 'Bach',
    'mixed' => array('Tomcat')
);
$dogs = array(
    'white' => 'Bowser',
    'brown' => 'Rex',
    'mixed' => ('Mutt')
);

var_dump('Cats', $cats, 'Dogs', $dogs);
?>

string 'Cats' (length=4)
array
  'white' => 
    array
      0 => string 'Kitty' (length=5)
  'brown' => string 'Bach' (length=4)
  'mixed' => 
    array
      0 => string 'Tomcat' (length=6)
string 'Dogs' (length=4)
array
  'white' => string 'Bowser' (length=6)
  'brown' => string 'Rex' (length=3)
  'mixed' => string 'Mutt' (length=4)

This snippet highlights what happens when the arrays being merged have matching keys.

1
2
3
4 <?php
var_dump('Cats and Dogs - plain', array_merge($cats, $dogs));
var_dump('Cats and Dogs - recursive', array_merge_recursive($cats, $dogs));
?>

All values that’s not already in an array will be placed in an array, and merged with other values with the same keys. Arrays are left as they are and then merged.

array
  'white' => 
    array
      0 => string 'Kitty' (length=5)
      1 => string 'Bowser' (length=6)
  'brown' => 
    array
      0 => string 'Bach' (length=4)
      1 => string 'Rex' (length=3)
  'mixed' => 
    array
      0 => string 'Tomcat' (length=6)
      1 => string 'Mutt' (length=4)

Compare this with the normal array_merge function, where the values of the latter arrays replace all the values with the same key.

array
  'white' => string 'Bowser' (length=6)
  'brown' => string 'Rex' (length=3)
  'mixed' => string 'Mutt' (length=4)

Values with numerical keys get added to the original array with a renumbering of keys. This is the same behaviour as that of array_merge.

Setting up the Swiftmailer Spooler

2012-12-20T00:00:00+00:00

The Swiftmailer library comes with a nifty feature where mails can be spooled to disk and sent later asynchronously instead of mailing them immediately. This results in quicker response times from your scripts, and by extension happier users. This post will explain how to set it up outside of the Symfony 2 framework.

Spooling the Mail

Swiftmailer defines a number of Transports with which mails can be sent. A common one is the Swift_SmtpTransport. To spool mails, we’ll use the Swift_SpoolTransport for the initial sending, and then still use Swift_SmtpTransport for the background sending. The following comes (with some changes) from a post in the Swiftmailer Google Group:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17 <?php
// Setup the spooler, passing it the name of the folder to spool to
$spool = new Swift_FileSpool(__DIR__ . "/spool");
// Setup the transport and mailer
$transport = Swift_SpoolTransport::newInstance($spool);
$mailer = Swift_Mailer::newInstance($transport);

// Create a message
$message = Swift_Message::newInstance('Excellent Subject')
    ->setFrom(array('sende...@domain.com' => 'John Doe'))
    ->setTo(array('your_emailAddress@domain.com'))
    ->setBody('spool messages !!!');

 // Send the message
$result = $mailer->send($message);

echo "SPOOLED $result emails";

This will spool the email to specified folder instead of sending it. At this stage, the actual sending of the mail hasn’t happened yet. You will need to flush the spooled messages using a cron or background script that calls the flashQueue method to do that.

Sending the Mail

The background script will use the Swift_SmtpTransport to send the spooled mails:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23 <?php
//create an instance of the spool object pointing to the right position in the filesystem
$spool = new Swift_FileSpool(__DIR__."/spool");

//create a new instance of Swift_SpoolTransport that accept an argument as Swift_FileSpool
$transport = Swift_SpoolTransport::newInstance($spool);

//now create an instance of the transport you usually use with swiftmailer
//to send real-time email
$realTransport = Swift_SmtpTransport::newInstance(
    "smtp.gmail.com",
    "465",
    "ssl"
)
    ->setUsername("username")
    ->setPassword("password");

$spool = $transport->getSpool();
$spool->setMessageLimit(10);
$spool->setTimeLimit(100);
$sent = $spool->flushQueue($realTransport);

echo "SENT $result emails";

And that’s it. All that’s left now is to call the background script periodically using something like cron.

Building PHP Projects with Jenkins

2012-06-09T18:46:23+00:00

It can become quite difficult to keep PHP code structured and clean. There’s a couple of good tools out there (PHP Code Sniffer, PHP Mess Detector, etc) to help you manage it, but it adds a lot of overhead that can bring your productivity down. One way to keep the advantage of checking your code without taking a productivity hit, is using a build system like Jenkins.

Look at this blog post to setup Jenkins and the various tools. There’s also a Jenkins PHP Template that will help a great deal in getting things started.

I’m planning to play around with it, and post updates here about how it goes.

Decorator Pattern Implemented Properly In PHP

2012-03-17T11:14:56+00:00

While working on Backend-PHP I needed to do a proper implementation of the Decorator Pattern in PHP. Just googling PHP decorator pattern will come up with a number of simple solutions, but none of them are usable in a general, robust way that a framework requires. So I extended and tweaked the implementations a bit.

##TL;DR

Most decorator pattern implementations for PHP found on the web is broken. They need some extra functionality to work properly, especially for nested decorators. See the complete solution in the Backend-PHP code

##The Basics

I’m assuming you know how decorators work. If not, read up on it first! I’ve implemented a base Decorator class to handle the base functionality for Decorators. The simple implementations out there serve as a good base for an initial implementation:

class Decorator
{
    protected $object; //The object to decorate

    function __construct( $object)
    {
        $this->object = $object;
    }
}

This is enough to get you started, but you will need to wrap the original object’s properties and methods manually. Let’s define a couple of magic functions to do that automatically:

public function __call($method, $args)
    {
        if (is_callable($this->object, $method) {
            return call_user_func_array(array($this->object, $method), $args);
        }
        throw new Exception(
            'Undefined method - ' . get_class($this->object) . '::' . $method
        );
    }

    public function __get($property)
    {
        if (property_exists($this->object, $property)) {
            return $this->object->$property;
        }
        return null;
    }

    public function __set($property, $value)
    {
        $this->object->$property = $value;
        return $this;
    }

That should give you a working basic implementation.

##More Advanced

That will work as long as you don’t nest decorators, ie, decorate an already decorated object:

$object = new DecoratorOne(new DecoratorTwo(new SomeClass()));

Why? The magic functions will check the decorated object for the properties and methods, but in the case of DecoratorOne above, the decorated object will be another decorator, and you won’t get the expected behaviour. This happens because property_exists doesn’t trigger the __get (or __set) methods. is_callable does check for the __call function, but that doesn’t necessarily produce the expected results, especially if you stack decorators that modify method execution.

We solve this by adding two more functions:

###getOriginalObject

public function getOriginalObject()
{
    $object = $this->object;
    while ($object instanceof Decorator) {
        $object = $object->getOriginalObject();
    }
    return $object;
}

getOriginalObject will return the original, undecorated object. This is handy to get the correct class name or access to undecorated methods and properties:

echo get_class($decoratedClass) //Will give the name of the Decorator

echo get_class($decoratedClass->getOriginalObject()) //Will give the name of the original object

###isCallable

public function isCallable($method, $checkSelf = false)
{
    //Check the original object
    $object = $this->getOriginalObject();
    if (is_callable(array($object, $method))) {
        return $object;
    }
    //Check Decorators
    $object = $checkSelf ? $this : $this->object;
    while ($object instanceof Decorator) {
        if (is_callable(array($object, $method))) {
            return $object;
        }
        $object = $this->object;
    }
    return false;
}

isCallable will check the original object for the specified method, and if not found, it will check the other decorators for the method. The option exists to check itself as well.

##The Result

Now we can modify the magic functions like so:

public function __call($method, $args)
{
    if ($object = $this->isCallable($method)) {
        return call_user_func_array(array($object, $method), $args);
    }
    throw new Exception(
        'Undefined method - ' . get_class($this->getOriginalObject()) . '::' . $method
    );
}

public function __get($property)
{
    $object = $this->getOriginalObject();
    if (property_exists($object, $property)) {
        return $object->$property;
    }
    return null;
}

public function __set($property, $value)
{
    $object = $this->getOriginalObject();
    $object->$property = $value;
    return $this;
}

This will give you a robust Decorator Pattern implementation from which you can then create more Decorators. See the complete implementation in the Backend-PHP code

The Low Hanging Fruit Of Testing

2012-03-16T15:14:03+00:00

I’ve been getting more into Testing (specifically Unit Testing) the past few weeks, and although I’m still struggling to firstly get it right, and secondly see the long term benefits, I know that the quality of my code is already better. Let’s call it the low hanging fruit of Testing.

I find that I’m reordering and sometimes rewriting functions, and sometimes whole classes, because just the need to test it already shows some design flaws. A class with too many private methods, a function with no clear input / output, all of that shows up when you start to write tests.

The fact that I have to think of my code and classes in terms of an API requires that I make sure of what the purpose of a function is, and what it’s signature and return should look like. Because it’s difficult to test large, complex functions, I’m refactoring those into smaller, more modular functions.

So I’m spending more time with the code, making improvements as I read and think about the code. So yeah, I’m not utilizing the full power of testing yet (not to speak of Test Driven Development), but I’m already enjoying the fruits.

json-encode'ing Private and Protected Properties

2012-02-24T06:51:15+00:00

When you json_encode an object, it will ignore all protected and private properties if the json_encode isn’t called from within the object. Sometimes you want to output the protected and private properties of an object when you’re JSON encoding it. Here’s how to do it in an easy and elegant way:

##The Problem

The properties of an object json_encode includes in it’s eventual output depends on the scope of the function call: If it’s from inside the object, it will include all the properties, private and protected. If it’s from outside of the object, it will only include the public properties.

1
2
3
4
5
6
7
8
9
10
11
12
13
14 <?php
class Tester
{
    public $public       = 'Public';
    protected $protected = 'Protected';
    private $private     = 'Private';
    function foo()
    {
        echo json_encode($this);
    }
}
$test = new Tester();
$test->foo(); //Outputs {"public":"Public","protected":"Protected","private":"Private"}
echo json_encode($test); //Outputs {"public":"Public"}

This isn’t necessarily the desired outcome. Sometimes you want more control over what is being included in the encoding function.

##The Solution

The easiest way to solve this is to add a _toJson function to your object, and call that instead of json_encode:

1
2
3
4
5
6
7
8
9
10
11 <?php
public function _toJson()
{
    $properties = $this->getProperties();
    $object     = new StdClass();
    $object->_class      = get_class($this);
    foreach ($properties as $name => $value) {
        $object->$name = $value;
    }
    return json_encode($object);
}

The getProperties function should return the properties you want to include in the encoding. At it’s simplest it can be

1
2
3
4
5 <?php
    public function getProperties()
    {
        return get_object_vars($this);
    }

Then, to call it, just check if the function exists, otherwise just call json_encode:

1
2 <?php
    echo method_exists($object, '_toJson') ? $object->_toJson() : json_encode($object);

It’s relatively simple to do it as a Decorator as well, which will enable you to give this functionality to multiple classes without the need to duplicate the code. See how I did it in the Backend-PHP framework.

Do you have a better way to do it? Say so in the comments.

22seven.com - Folly, or Genius?

2012-01-28T08:37:06+00:00

##Promising Product

Last week Christo Davel, founder of the now defunct twenty20 bank, released an online personal accounting service called 22seven.com. It looked like a great product. Something I was looking for for quite a while: A simple online interface that automatically imports my transaction details, and reports on how much I’m spending on what.

I will never use it, though. Why? The app asks for your online banking login details.

##But…

In case you don’t know. This is “not a good thing”(tm). As tweeted by Micheal Jordaan, CEO of FNB, you’re taking a big risk by doing so. Once you’ve given it to them, it’s out there, somewhere. Yes, they secure the information, but to use it, it needs to become readable at some point. If it can become readable, someone can steal it. From a dishonest employee, to a hacker that found his way on to their servers.

The question is, why did they decide to go the route of asking for a user’s banking login details? The banks have been struggling to educate users so that they don’t enter the online banking sites from any other place than the bank’s site, and that they don’t give their login credentials to any third party. As far as I know, the South African banks will refund any money you lost because of online fraud, except if they can prove that you gave your credentials to someone else. And this service is asking for it, wholesale.

##The Problem

The problem is that this isn’t the only company that went this route. Sidpayment, a product from Setcom, gives merchants the ability to do online EFTs. The problem is that it’s a Java app that looks over the customer’s shoulder while he’s doing a once off payment through his internet banking. The opportunity for hacks and attacks abound. I’m sure both these knew the risks and were aware of the potential backlash, but why did they knowingly proceed with it?

There’s a simple answer: They had no choice. The banks in South Africa are walled gardens, with all of your information locked up in it. There’s no way to access it, except through their portals: the branch and online banking. Even FNB, who seems to be embracing newer technologies, doesn’t have anything that even looks like an API. All they have is an accounting app that looks like it was written by (no offence) accountants. So how are third party applications supposed to access a client’s transaction details?

Is there even a need for third parties to access people’s transaction information? Of course there is! In a number of other countries you see innovation in personal finance because they have access to information. There’s a number of services that track your credit card and spending patterns. These services depend on (sometimes read-only) data they get from banks and financial institutions.

##Come on, Banks!

If we are to see innovation and progress in personal accounting in South Africa, the banks need to lead the way by not walling everything off, but by giving accredited and approved third party developers and companies access to API’s. Perhaps 22seven.com’s play was to force the hand of the big banks, and not as stupid as it seems.

Setting up Puppet on Ubuntu 10.4

2011-11-16T04:24:06+00:00

I’m trying to use puppet to setup standard dev and (eventually) production environments. As Ubuntu 10.4 is the current LTS release (at least until 12.4), I need to get puppet running on 10.4. The problem is that although the latest stable release for puppet is 2.7.6, puppet in the ubuntu repo is version 0.25.4. Here’s how to firstly set it up, and then upgrade it to the latest version.

Setup

First install packages required by Puppet via aptitude

sudo apt-get update
sudo apt-get install irb libopenssl-ruby libreadline-ruby rdoc ri ruby ruby-dev

And then install puppet and puppet master, also through aptitude

sudo apt-get install puppet puppetmaster

You can skip install puppet if your working on the master machine, or skip puppetmaster if your working on a client. I’ll be installing the client and the master on the same machine.

You can then follow the following steps from the documentation to get the default setup going. Run all commands as root:

#Generate a default config file
puppetd --genconfig > etc/puppet/puppet.conf
#Generate a default site manifest
puppetd --genmanifest > etc/puppet/manifests/site.pp
#Add the puppet user and group
puppetd --mkusers

We also need to tell the client which server it should use as it’s master. Add the following line to /etc/hosts:

127.0.0.1 puppet

Puppet uses the host name puppet as the default master. You can change this by setting the server config value in /etc/puppet/puppet.conf

You can then test your client setup by running

puppetd --test

Cert Issues

At this point I ran into a stupid config issue, where the client complained bombed out with the following error:

could not retrieve catalog from remote server: hostname was not match with the server certificate

Despite the bad english and uninformative message, I eventually figured out that even though a puppet client by default uses the puppet hostname as its master, the name on the cert that was generated for the hostname isn’t puppet, but the current machine’s name. This mismatch causes the confusion, so either change the server config value, or regen the cert with hostname puppet. I just changed the config value to my machine’s name. This actually negates the need for the added line in /etc/hosts.

PHP Community in South Africa, lack thereof

2011-10-05T05:07:46+00:00

I really like working with PHP. It’s a great language despite, a number of shortcomings, and I get to make a living out of writing PHP code. I also love to socialize, and the past few weeks I’ve been looking for a way to combine these two.

I’ve looked for conferences or get togethers that I can attend. I looked for online groups or chatrooms about PHP where South Africans gathered. But I couldn’t find any. Well, any active, constructive ones in any case.

I see that there’s an active Drupal community in and around Jo’burg, so I’ll try to attend some of their meetups.

Why Johnny Died (or, How I Finally Realised Why Singletons And Global Variables Are Bad)

2011-09-28T04:06:19+00:00

Johnny (an asthma sufferer and fictitious character) died yesterday. He got an asthma attack, but he could not find his inhaler, and sadly passed away. Usually his mother kept it for him and made sure he used it, but on the first day he ventured out alone, he got an asthma attack, realised that he didn’t have his inhaler with him, and died. What does this have to do with Singletons or Global variables? Read on…

I’ve been struggling to get my head around why singletons and global variables are bad. Particularly in the context of unit testing. I’m quite a fan of a global static class that can manage certain resources for you. But if you just look around a bit, you’ll find a few resources around the fact that they are a bad idea.

I read up quite a bit on why exactly globals and singletons are bad, but couldn’t find any convincing explanations. Until yesterday, when I read this question and answer on stackoverflow. Particularly

every class can potentially depend on a singleton, which means the class can not be reused in other projects unless we also reuse all our singletons

In other words, if you want your classes / objects to be modular and easily transportable and reusable in other code bases, you cannot use global variables or singletons. To properly unit test, you need to use your class in a clean environment, with only it’s stated dependencies injected into it on creation. Otherwise there’s no guarantee that your tests will fail or succeed consistently. Doing this also promotes code reuse, as you know that the global state won’t be affected in some averse way.

What does this have to do with Johnny? Look at him as an object of the class Human, who has a dependency on an Inhaler object. His mother is a global variable or class that supplies him with the Inhaler when he needs it. All fine and dandy if Johnny operates within the context of his mother’s home. Take him out of that context, and he dies when he needs the inhaler. Rather give him the inhaler, and let him keep it in his pocket. Dependency injected!

A PHP Virtual Host

2011-09-13T03:18:03+00:00

I hate repetitive work. I love solving a puzzle. So it often happens that I’d spend 2 hours writing a script to solve a repetitive problem, than just taking the 2 minutes to do it manually. Creating apache config files are tedious and boring, so I wrote a PHP script that emulates Virtual Hosting.

You can get the code on GitHub. A Warning: The code is not production ready. Use it at your own risk. Here’s what it does:

An .htaccess file redirects all requests to the index.php file, which sits in /var/www. We get the host from the $_SERVER variable, and check if a folder for that host exists.

1
2
3
4
5
6
7
8 <?php
$host   = $_SERVER['HTTP_HOST'];
$folder = getcwd() . '/' . $host . '/';
if (!file_exists($folder)) {
no_host();
die;
}
?>

So if we wanted to server jrgns.net from it, the folder /var/www/jrgns.net should exist on the server. If it doesn’t, a generic message is displayed.

Courtesy of the .htaccess file, the whole URL, except the host name, gets passed to the script in the f query variable. As we might be requesting a stylesheet such as jrgns.net/styles/basic.css, the virtual host should pick that up. If no file is requested, check for and serve an index file.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29 <?php
if (empty($_REQUEST['f'])) {
	$indexes = array(
		'index.php', 'index.html', 'index.htm', 'main.php'
	);
	foreach($indexes as $index) {
		if (file_exists($folder . $index)) {
			chdir($folder);
			include($folder . $index);
			die;
		}
	}
} else {
	$file = $_REQUEST['f'];
	if (file_exists($folder . $file)) {
		$info = explode('.', $file);
		switch(end($info)) {
		case 'js':
			header('Content-Type: text/javascript');
			break;
		case 'css':
			header('Content-Type: text/css');
			break;
		}
		readfile($folder . $file);
		die;
	}
}
?>

I use a very rudimentary way to get the file extension, and then send the content header. There are probably better ways to do this, but it works. The rest is just the generic message that gets displayed if we don’t know what you’re looking for.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19     <?php
    function no_host() {
    	global $host;
    ?><!DOCTYPE html>
    <html>
    	<head>
    		<title>Unknown Domain: <?php echo $host ?></title>
    	</head>
    	<body>
    		<div>
    			<h1>Unknown Domain: <?php echo $host ?></h1>
    		</div>
    	</body>
    </html>
    
    <?php
    }
    no_host();
    ?>

Like I said. Simple!

A JavaScript trim function

2011-07-18T04:55:32+00:00

A simple enough trim function for JavaScript:

function trim(string) {
    return string.replace(/^\s*|\s*$/g, '')
}

The regular expression translates as

^ - At the beginning of the line
\s* - Take all the white space you can find, if there is any
| - OR
\s* - Take all the white space you can find, if there is any
$ - At the end of the line

The g parameter at the end ensures that all instances are replaced.

The replace function uses the regular expressions to replace the white space it found at the beginning or the end of the string with an empty string.

Easy enough!

Quick Guide To Creating A Website Quickly

2011-07-11T21:31:53+00:00

The last two weeks I had the privilege of staying in a house on the beach near Mosselbay. This time of the year whales start coming into the bay, and, if you’re lucky, you can spot them breaching or displaying their flukes or fins. This inspired me to create WhaleSpotter, a site that uses Twitter to help people spot whales along South Africa’s south coast. It took me less than 2 hours to do it. Here’s how.

###Know exactly what you want to Do

This speaks for it self. Spend a few minutes to lay out what you want, and how you want to achieve it. If you need to write it down, explain it to someone or whiteboard it, do it. This will save a lot of time later in the process, as you won’t track back or waste time with features or ideas that doesn’t fit.

###Keep it Simple

Keep the number of features down to a minimum. If you can fit everything on one page, do. This is quick and dirty, not long and elaborate. You can expand it later. Your aim now is to get something up and running.

###Know and use your Tools

Frameworks cut down a lot in development time. They provide functionality and structure so that you don’t have to work on it. I dev’ed my own MVC framework, Backend, so I used that. It provides a nice templating system, along with basic configuration, caching and utility functions. Use what ever you’re comfortable with, as long as it reduces the work you have to do, not increase it.

###Use Resources wisely

Other resources can also cut down on time taken to get your site up. I used Google Images to get a logo. Look at Themeforest for a design that will work for your site. Copy and paste code from your own projects or open source projects on the web. The site was in part inspired by http://replaceawordinafamousquotewithduck.com/, and I copied some of the twitter parsing code from the GitHub repo.

###So

Know where you’re going. Keep it simple. Know your tools and options. Outsource as much as possible.

Good Developers Think like a Startup

2011-05-05T08:25:54+00:00

I have the privilege to be the Head of Development of Branded Internet, a white label ISP in South Africa. I also like to read and think about how startups work, and what they do to make them successful. As I see it, the basic recipe for a startup is:

Find a niche market or problem
Find a revolutionary / novel way to solve that problem
Build the solution
Market it
See the money roll in.

If you see the company you’re developing for as your client or target market, software development follows basically the same recipe.

Find a problem

Usually people come to the developers of a company with a problem. I need App X to do Y and Z. App C doesn’t do D correctly. I’m not very good at spotting problems that are potentially money making, so I’m quite happy to be a developer where people come to me looking for solutions. That’s the first step sorted.

Find a way to solve the problem

Startups depend on the fact that they solve previously unsolved problems, or they solve problems in such a way that people want to throw money their way for the privilege to use the startup’s solution. I suspect that this may just be one of the reasons why developers at bigger companies don’t necessarily produce quality code: There’s no incentive to do it quickly, elegantly or intelligently. Just get it done. If it takes you three times longer than projected with a code base larger than the Kalahari, who cares. It’s done, and you got your salary at the end of the month.

If a startup thinks like that, it’s dead in the water.

Solving problems better is always better. Yes, there’s major time and money constraints on most developers, and elegance isn’t always achievable, but good developers should always be open to solve problems in a novel way. Don’t be a mill churning out code, be a creator of great code!

Build the Solution

Do what you do. Be a thinker, then a typist. And do it well.

Market it

Having built the solution is not where it stops. For most developers, it’s where they wished it stopped, because, let’s face it, if we liked speaking to people, we wouldn’t have become developers. But unfortunately you as a developer need to take it further. You need to market what you’ve done. A product that no one knows or knows how to use doesn’t exist.

You need to tell your superior and your peers about the fact that the project is up and running. You need to document the code and the use cases so that other developers (including your future self) knows what’s going on. You need to train your clients (the users) who originally requested your help on using the project. You need to get feedback from your clients on if this solves their problem, and go back to the code if it doesn’t.

Reap the Rewards

I find it immensely rewarding to be able to solve problems for people. Yes, the salary is great, but there’s times that I will literally code for free. Not just because I like solving problems, but because I know that I can make someone’s work day less frustrating. You get the gold and the golden glow.

tl;dr

By thinking or your rote work in terms of a startup, where you need to be agile and more than just a normal developer, going out of your comfort zone and speaking to people, your average work can become great work, and your worth will be measured in more than just a salary check.

Why Object Orientation is Great

2011-02-08T07:00:21+00:00

I’m working on some statistics surrounding BI’s API Documentation coverage: Just how much user documentation have we written?

When I started out, the number was abysmally low: 11% of the 1300 functions. Not good. But I didn’t feel too bad when I realized that because it’s written as an Object Orientated system, I can very quickly improve the coverage.

As I went through the undocumented functions, it became clear that a lot of them are defined in some of our base classes. So, by documenting them in the base classes, I will immediately improve the documentation coverage for a whole bunch of classes. I tried it with one function, and lo and behold, the percentage jumped with 6 points!

So, what do we learn from this? Object orientation is powerful when you implement changes deep down in the object stack, as it propagates through the whole stack. It also reduces the work load (most of the time) as there’s no need to repeat yourself.

The downside to consider is that the documentation written is very general (as it should be in a base class), and will probably be overwritten in some of the sub classes.

But that’s Object Orientation: Solve 80% of the problem with 20% of the code, and the other 20% of the problem with the other 80% of the code!

Getting Started with node.js

2010-10-11T06:31:40+00:00

Looking for a new challenge, I started a small project in node.js. Here’s a few simple tips on how to get going quickly.

It’s JavaScipt

Yeah, that’s obvious, but sometimes stating the obvious is necessary. Node.js is essentially server side JavaScript. This means that all your JS knowledge and tutorials and manuals will be usefull once you get stuck.

It’s Event Driven

Node.js is an event driven framework, which can take some getting used to, but it is very powerful in the long run. You can write straight forward bottom to top code, but then you’ll be missing out on the real power of node.js. An example:

//Get the events module
var   events = require('events')
    , fs     = require('fs');

//Create a new event object
var file_emitter = new events.EventEmitter();

//Create a listener for our event object
var newFileListener = file_emitter.on('found', function(file) {
    //Do something to the found file
}

//Check for files in a folder
var files = fs.readdirSync('/some/folder');
files.forEach(function(file) {
	//Emit (Trigger) the "found" event on our event object
	file_emitter.emit('found', dest);
});

Multiple Files

Any good coder knows that putting your code in multiple files is a quick win for organized coded. Here’s how to do it in node.js:

//In utils.js
exports.trim = trim;
function trim(string) {
	return string.replace(/^\s*|\s*$/g, '')
}

//In the file you want to use the trim function
var utils = require('./utils')

The .js part of the file is automatically added/detected. This is to include a file in the same folder as the calling file, but you can easily just add the correct path traversals to get to other files. The important part in the file being included is setting the function name in the exports Object.

File I/O

Files are all treated as streams in node.js. This means that you can start acting on the data in a file before node has even finished reading the file.

//Open a file as a Readable stream
stream = fs.ReadStream(file);
stream.setEncoding('ascii');
stream.on('data', function(data) {
    //Do something to the file's data mid stream
});
stream.on('close', function () {
    //Do something to the file once it's finished reading
}

Writing to a file is just as simple

stream = fs.createWriteStream(filename, { 'encoding': 'base64' });
stream.write(data);
stream.end();

Want More?

See parser_email on github for the full source of the examples given.

Some useful sites:

Email Parser for node.js

2010-10-07T09:53:55+00:00

I need to parse Emails in node.js and after looking around on the web, I didn’t find much. Oh, no, I’ll have to write it myself… :)

The source is hosted on GitHub .

Here’s an example:

var   fs        = require('fs')
    , sys       = require('sys')
    , em_parse  = require('./parser_email')

stream = fs.ReadStream(file);
stream.setEncoding('ascii');
stream.on('data', function(data) {
	mail += data;
});
stream.on('close', function () {
	parser = em_parse.parser_email();
	parser.setContent(mail);
	parser.parseMail();
});

Why I Code

2010-04-15T08:56:23+00:00

A a student I went to a career psychologist to explore my career needs and drives, and got told that I like solving problems in a creative way. As the son of an engineer and a very artsy mother, that kind of made sense. Seeing as I was already into computers and programming, I chose coding as my problem solving mechanism.

Through the years I learned that I loved, never mind liked, solving problems this way. I wrote a framework, because I saw a lot of unstructured and repetitive code in my projects. I dev’ed What’s up in Melville because I needed information about the area I lived in. I saw Hacker News, and wanted the same site, but for South Africa, so I dev’ed ZA Coders. All (hopefully) creative solutions to problems I faced.

I am now Head of Development at Branded Internet, and I’m still very passionate about solving problems through code. Any dev department at a company whose main aim is not to produce software, is to help the rest of the company to solve their problems through code. Finances needs to regularly generate a certain report, Dev scripts the code, and gives them the link. Support needs to information on an account, Dev scripts the code, and gives them the link.

Sometimes Dev departments will write code to help themselves, but most of the work will be done for other departments. So the Dev department needs to communicate, needs to interact, so that they know which problems to solve to help the other departments do their job.

Development departments exists to help other departments. In other words, it’s not about me.

Twitter API Request Entity too large / HTTP 413 error

2010-03-30T07:12:09+00:00

I’ve setup zacoders.net that it will automatically tweet new RFC’s. A while back this functionality stopped working. Today I investigated a bit further, and here’s what I found.

When posting an update, the Twitter API responded with a 413 HTTP error code, and the following text:

Request Entity Too Large

The requested resource

/1/statuses/update.json

does not allow request data with POST requests,
or the amount of data provided in the request
exceeds the capacity limit.

I realized that the backend’s OAuth utility sent through POST requests with the parameters included in the URI, just like a GET request:

POST HTTP/1.1 http://api.twitter.com/1/statuses/update.json?parameters=here

What baffled me is that I only got this error when running it on my live server, not on my testing local server. I suspect it might be a difference in the Curl libraries, but I’m not sure.

To fix it, I just updated the code to only add the parameters to the URI when it was a GET, and to POST the data when it was a POST, and voila, it’s working again.

How to remove the www from your URL

2010-03-26T08:03:29+00:00

If you like simplicity, you might like to simplify your website’s URL. You can do this by adding the following in your .htaccess file, or in the VirtualHost declaration on your Apache setup.

RewriteEngine on
RewriteCond %{HTTP_HOST} ^www.mysite.co.za$ [NC]
RewriteRule ^(.*)$ http://mysite.co.za/$1 [L,R=301]

The rewrite module for apache needs to be installed for this to work.

Remember that you can tell Google to list all links to your site with / without the www by setting that in the Webmaster Tools.

New look!

2010-03-24T23:05:40+00:00

At long last I’ve been able to find the time to port my website from drupal to backend. Backend is a PHP framework I wrote myself and open sourced to get it out there.

I’ve also redesigned the site, made it simpler and hopefully more usable. I’ll be adding more information as I go along.

SQLite Viewer

2010-03-18T16:28:30+00:00

I’m playing around with Google Gears, doing data imports, and generally playing around. I’m quite visiually oriented, so I need to SEE that the data has been added to a table. For my conventional PHP / MySQL apps, I usually use phpMyAdmin or the MySQL query browser, but for Gears developmen, which uses SQLite, I needed to use something else.

I wrote a small JavaScript / Gears app which enables you to view the contents of a SQLite database. All you need to use it, is to download the file, and unzip it into the folder where your web app sits. And no, you won’t be able to use it for a remote DB. You need at least FTP access to the website you want to look at, and it will only show you your local data.

You will also need to know the name of the database. At the moment I can see no simple way to supply the user with a listing of DB’s, so the app prompts the user for the DB name.

The app currently links to the scripts and styles on my host, which is fine for light use. If you plan to use this app extensively, please start hosting the files on your own host and update the HTML!

Update On Backend

2010-03-18T16:23:37+00:00

Bakend: PHP framework, as it’s now called, now has it’s own website! Check it out at http://backend-php.net.

Follow backend on Twitter at @backend_php!

Backend 100 Revisions

2010-03-18T16:20:29+00:00

I’ve been working on backend, a PHP framework the last couple of months, and it’s really starting to take shape. I’ve learned a lot about what one can do by coding smarter and the advantages open source code gives on.

After getting somewhat disheartened a few weeks back, I decided to take a look at how CodeIgnitir does things, and wow, it blew my mind. After spending some time with their code base, I had so many ideas I wanted to implement in backend and my thoughts around it was instantly rejuvinated. Problems that stopped me from progressing had new solutions, and the momentum I gathered from that helped to solve some problems that didn’t have any solutions directly at hand.

So, backend just hit it’s 100th revision, somewhat arbitrarily, as I reset the branch at some point, and don’t always follow the exact same commit methodology, but hey, it looks like a milestone, I’ll treat it as one.

What has changed?

URL mapping has improved a lot (thanks to CodeIgnitir). ?q=controller/action/param1/param2 now maps to Controller::action(param1, param2) without much effort. This has also helped to work towards being a RESTful API.

The installation process is now very smooth and stable. The setting up of data sources need some work, but once that is done, getting the application up and running and fine tuning it is a breeze.

The beginning of a Links structure with named link lists can be managed. The primary use will be for primary and secondary links for a website.

I’ve managed to implement tags for content, as well as RSS feeds for content as a whole, or content in a tag. The ease with which I could do it really gives me hope that backend will make my coding life easier!

All around polish and stability. Especially around the HTML templating and rendering engine.

Where to from here?

The permissions / security infrastructure still needs some work. I’ve managed to automatically install most of the default permissions, but there’s no useful management interface yet.

I want to start using backend on my personal site (this one) and on another site I’ve got in the works, called zacoders.net. More on that later.

My take on single or multiple returns

2010-03-18T16:17:11+00:00

Originally posted on StackOverflow

I force myself to use only one return statement, as it will in a sense generate code smell. Let me explain:

<?php
function isCorrect($param1, $param2, $param3) {
    $toret = false;
    if ($param1 != $param2) {
        if ($param1 == ($param3 * 2)) {
            if ($param2 == ($param3 / 3)) {
                $toret = true;
            } else {
                $error = 'Error 3';
            }
        } else {
            $error = 'Error 2';
        }
    } else {
        $error = 'Error 1';
    }
    return $toret;
}
?>

(The conditions are arbritary…)

The more conditions, the larger the function gets, the more difficult it is to read. So if you’re attuned to the code smell, you’ll realise it, and want to refactor the code. Two possible solutions are:

Multiple returns
Refactoring into separate functions

Multiple Returns

<?php
function isCorrect($param1, $param2, $param3) {
    if ($param1 == $param2)       { $error = 'Error 1'; return false; }
    if ($param1 != ($param3 * 2)) { $error = 'Error 2'; return false; }
    if ($param2 != ($param3 / 3)) { $error = 'Error 3'; return false; }
    return true;
}
?>

Separate Functions

<?php
function isEqual($param1, $param2) {
    return $param1 == $param2;
}

function isDouble($param1, $param2) {
    return $param1 == ($param2 * 2);
}

function isThird($param1, $param2) {
    return $param1 == ($param2 / 3);
}

function isCorrect($param1, $param2, $param3) {
    $toret = false;
    if (!isEqual($param1, $param2)
        && isDouble($param1, $param3)
        && isThird($param2, $param3)
    ) {
        $toret = true;
    }
    return $toret;
}
?>

Granted, it is longer and a bit messy, but in the process of refactoring the function this way, we’ve

created a number of reusable functions,
made the function more human readable, and
the focus of the functions is on why the values are correct.

Firefox requests a page twice?

2010-03-18T16:13:43+00:00

I’m busy creating a site for a client who wants to know when certain internal links get clicked. Not a problem, I just insert a record into a table every time the links they want to know about get accesesd. Only problem is, when I check the table, I notice that every page impression gets logged twice.

Now, this happened before, where I also needed to implement some page tracking, and it ended up being an image or something with src=””, and Firefox interpreting it as the same location as the current page (as with form with action=””). Don’t know if that is according to standards, either way, it messes up my page tracking.

The simple solution is to find any images or HTML elements with source attributes that are empty. Only problem is I couldn’t find any this time. So, by taking out chunks of code, I eventually tracked it to srcover attributes in some of the images. Just as a side note, I don’t use srcover, the designers use them, and unfortunately I must make do with what the designers give me.

So why does Firefox (I didn’t test on other browser) do this on srcover? Firebug shows that the images specified in srcover are downloaded, and the mouseovers work, so it’s not that the images are missing or anything.

I’ll investigate when I have some time, and report back…

Update:

I found this page at Experts Exchange, but hey, they want me to pay to see it. No chance of that happening. If anyone is a member, and you don’t want money to share some wisdom, please do!

Update number two:

I fount this page which only talks about the empty src tag, but it also quotes the HTML spec that says that empty src tags refer to the same page their in. So why does srcover still re-request my page?

Update number final:

Ok, so it wasn’t Firefox’s fault, but the javascript that did the rollover images. When it preloades, it doesn’t check for empty src attributes, it just loads them. As it checks for both srcover and srcdown, which aren’t necessarily specified, it gets quite a lot of empty tags. I guess that Firefox uses a cached page for the rest of the requests.

The javascript library is called imagerollover.js by Adam Smith. I couldn’t find a working website, so I don’t know if I have the latest version, but I’ll upload my modified version…

Parse Http Accept Header

2010-03-18T16:10:35+00:00

I want to generate CSS on the fly for a personal project. I’m doing this by redirect all requests for files that does not exist to index.php, and then, if the request is for a stylesheet, index.php generates it.

To identify a request for a stylesheet, we need to parse the Accept header in the HTTP Request. I’ve grouped all the functions together in a Parser class.

This function retrieves the header, and then determines the mime type, precedence, and tokens for each type given. It’s then sorted according to RFC 2616 specifications.

<?php
public static function accept_header($header = false) {
	$toret = null;
	$header = $header ? $header : (array_key_exists('HTTP_ACCEPT', $_SERVER) ? $_SERVER['HTTP_ACCEPT']: false);
	if ($header) {
		$types = explode(',', $header);
		$types = array_map('trim', $types);
		foreach ($types as $one_type) {
			$one_type = explode(';', $one_type);
			$type = array_shift($one_type);
			if ($type) {
				list($precedence, $tokens) = self::accept_header_options($one_type);
				list($main_type, $sub_type) = array_map('trim', explode('/', $type));
				$toret[] = array('main_type' => $main_type, 'sub_type' => $sub_type, 'precedence' => (float)$precedence, 'tokens' => $tokens);
			}
		}
		usort($toret, array('Parser', 'compare_media_ranges'));
	}
	return $toret;
}
?>

This helper function parses the options for each mime type, determining the precedence and tokens. It returns the precidence as a float, and an array of tokens.

<?php
public static function accept_header_options($type_options) {
	$precedence = 1;
	$tokens = array();
	if (is_string($type_options)) {
		$type_options = explode(';', $type_options);
	}
	$type_options = array_map('trim', $type_options);
	foreach ($type_options as $option) {
		$option = explode('=', $option);
		$option = array_map('trim', $option);
		if ($option[0] == 'q') {
			$precedence = $option[1];
		} else {
			$tokens[$option[0]] = $option[1];
		}
	}
	$tokens = count ($tokens) ? $tokens : false;
	return array($precedence, $tokens);
}
?>

This function is used to sort the array of mime types we parsed from the header. At the moment it doesn’t check any tokens, but this code can easily be added on line 6.

Backend-PHP Framework

2010-03-18T08:46:32+00:00

Update: 21/11/2011

Backend has undergone a major rewrite. Get the latest info here, and get the code on GitHub

In the past few years I’ve been working on a PHP framework / backend. Sometimes out of frustration with copying and pasting code, but usually because I’m to lazy to write a lot of code, so decide to write a little bit of code to do a lot… And then I write some more code, to do a little bit more, and before you know it, it’s yet another PHP framework. I tried to get some public input on the project once before, but that went nowhere, and I left it there - until of course I had to do some work that required the same type of thing, and now I’m at it again.

What makes backend different from my previous failed experiment? Well, the fact that this is now the 7th or 8th time that I’ve rehashed the concept, and that this time, I’ve actually put some thought and effort into it. No, it’s not perfect, and it won’t solve the world’s problems, but it better and simpler, and hopefully more useful.

One concept that I definitely want to run with in backend, is the concept of backend as a web service, as opposed to a framework or a content / data management system. Backend provides a client, as in a browser or an application, with the requested data, in a format that the client can understand. Be it a human readable web page, JSON, XML or even PHP variables. This struck me as important after reading this by Ryan Tomayko:

Wife: A web page is a resource?

Ryan: Kind of. A web page is a “representation” of a resource. Resources are just concepts. URLs—those things that you type into the browser…

Wife: I know what a URL is..

Ryan: Oh, right. Those tell the browser that there’s a concept somewhere. A browser can then go ask for a specific representation of the concept. Specifically, the browser asks for the web page representation of the concept.

Wife: What other kinds of representations are there?

Ryan: Actually, representations is one of these things that doesn’t get used a lot. In most cases, a resource has only a single representation. But we’re hoping that representations will be used more in the future because there’s a bunch of new formats popping up all over the place.

Wife: Like what?

Ryan: Hmm. Well, there’s this concept that people are calling “Web Services”. It means a lot of different things to a lot of different people but the basic concept is that machines could use the web just like people do.

So it’s important for me to make it easy for web developers to easily provide data in different formats. Backend can already provide any client with data in 4, yes 4, different formats:

HTML, or a human readable webpage
JSON, the new darling of AJAX fans
(PHP) Serialized byte stream representations
PHP variables, as created by var_export

This will hopefully increase backend’s usefulness as a web service. And yes, I know, XML is missing, but of those 4, HTML was necessary, although not necessarily easy, and the other three were extremely easy to implement, as they are all products of simple PHP functions. One day when someone uses backend and actually needs XML, we can implement it. (Notice that I said we, not I - I rarely use XML, so I won’t know the requirements, so if I do it alone, it will probably be useless).

At the moment backend is on version 0.1.2.1 in my personal repo. All that means is that I’ve decided I’ve reached a point where I can actually release the code to the public, and made some improvements, fixes and changes. When will it be marked as version 1.0?

When we have the following:

A fully fledged and integrated admin space 2, Easy and intuitive setup / install system
Easy and intuitive update system
An intuitive plugin system
Intuitive and function DB Objects<
The ability to easily implement different data formats
The ability to use multiple and different data sources (MySQL, SQLite, flat files, etc.)
Working and secure user access control
Either a REST-full implementation, or a roadmap towards it
Something that makes coding easier

That’s ten requirements for me to release backend v1.0…

What’s already done?

Semi working DB Objects
Output of different data formats using views
Simple templating / caching system
Simple user access control
CRUD, actually, just CRU actions. I haven’t gotten round to the Deletes yet.

I’m using jQuery as a JavaScript backend, and blueprint for my CSS needs. For now it’s under the EPL, but I may switch to the MIT license at some point. I’m using MySQL exclusively at this point, but am hoping to add support for other DB’s / sources eventually.

I’ll be putting the code on Launchpad soon, so if you want to contribute, or comment, please do!

Importing a file into a Google Gears DB using PHP

2010-03-18T08:45:19+00:00

I’m playing around with Google Gears, particularly the database part, just to see what it’s usefull for, and if I can use it in an app I’m working on. At some point I realised that I will need to import data at some point. Here’s how I did it:

By using an HTML uploading form, and PHP to process the upload, you can create a JSON object which can be used to import data into whatever Gears DB you have. Use whatever you use to put all of the data into one array. The source can be a DB or a file upload, come to think of it. Convert the data to JSON using json_encode, and assign it to a JavaScript variable:

var data = ;

After that, check that the table exists, loop through the data, and insert!

The complete code below:

The test data to import:

"one two",2,1
"three four",3.4,2

And the test page, HTML and PHP together:

<?php
$data = false;
if ($_SERVER['REQUEST_METHOD'] == 'POST') {
	$data = array();
	if (!empty($_FILES['to_import']['tmp_name']) && is_readable($_FILES['to_import']['tmp_name'])) {
		$fp = fopen($_FILES['to_import']['tmp_name'], 'r');
		while (($row = fgetcsv($fp)) !== false) {
			$data[] = $row;
		}
		fclose($fp);
	}
	$data;
}
?>
<html>
	<head>
		<title>JSON / Gears import Test</title>
	</head>
	<body onload="do_import()">
		<form method="post" action="" enctype="multipart/form-data">
			<input type="file" name="to_import">
			<input type="submit">
		</form>
		<script src='gears_init.js'></script>
		<script>
var data = <?php echo json_encode() ?>;

function do_import() {
	var db = google.gears.factory.create('beta.database');
	db.open('database-test');
	db.execute('CREATE TABLE IF NOT EXISTS Import' +
				' (Phrase text, Amount float, Something int)');
	if (data) {
		for (var i = 0; i < data.length; i++) {
			db.execute('INSERT INTO Import Values(?, ?, ?)', data[i]);
		}

		rs = db.execute('SELECT * FROM Import');
		while(rs.isValidRow()) {
			alert(rs.field(0) + ' - ' + rs.field(1) + ' - ' + rs.field(2));
			rs.next();
		}
		rs.close();
	}
}
		</script>
	</body>
</html>

Redirect All Requests To Index.php Using .htaccess

2010-03-18T08:42:15+00:00

In one of my pet projects, I redirect all requests to index.php, which then decides what to do with it:

###Simple Example

This snippet in your .htaccess will ensure that all requests for files and folders that does not exists will be redirected to index.php:

RewriteEngine on
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule . index.php [L]

This enables the rewrite engine:

RewriteEngine on

This checks for existing folders (-d) and files (-f):

RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f

And this does the actual redirecting:

RewriteRule . index.php [L]

###Extended Example

You can extend this to pass the requested path to the index.php file by modifying the RewriteRule to the following:

RewriteRule ^(.*)$ index.php?q=$1 [L,QSA]

The ^(.*)$ part tells the rewrite module that we want to pass down the whole requested path as one parameter.
The QSA part tells the module to append any query strings to the request.
The ?q=$1 tells the module how to pass down the parameter. In this case, it’s passed down as the q parameter.

You can extend this even further by using regular expressions. For example:

RewriteRule ^([^/]*)(.*)$ index.php?first=$1&second=$2 [L,QSA]

This will pass down the first part of the path as the first parameter, and the rest as the second. So the following request

http://yourhost.com/some/path/somewhere

will result in

http://yourhost.com/index.php?first=some&second=path/somewhere

This allows for some creative ways to do clean URLs.

###Trouble Shooting

If it’s not working, make sure that mod_rewrite is installed on Apache. On a unix system you can just do

sudo a2enmod rewrite

to achieve that.

Jurgens du Toit 2017-02-02T05:34:34+00:00 http://jrgns.net/ Jurgens du Toit jrgns@jrgns.net 2017-02-02T05:34:34+00:00 http://jrgns.net/2017/02/02/2011-10-14-backend-core-a-restful-mvc-php-framework I like frameworks. I like working with them, I like writing them. I wrote <a href="http://backend-php.net">Backend-PHP</a>, which I’m using personally and at <a href="http://www.brandedinternet.co.za">Branded Internet</a>. Unfortunately it has a number of major flaws: It’s not easily unit testable, not really a REST web service, and it doesn’t conform to any popular coding standards. So I wrote <a href="https://github.com/jrgns/backend-core">Backend-Core</a> as the first step towards addressing these issues. <h2 id="minimal">Minimal</h2> Backend-Core is only intended as a minimal, core system. It doesn’t implement any application logic or data source abstraction. I decided to focus on the core functionality and design of a RESTful MVC system, and make sure that it works as it should. Eventually I’ll extend it with a seperate code base that will include application and data source logic. <h2 id="mvc">MVC</h2> The system is structured into Model, View and Controller components: <ul> <li>Model - The model contains all the business logic for an application. As this is a core system, the models are empty.</li> <li>View - The view determines how the results of a query will be displayed.</li> <li>Controller - The controller contains the application logic. Once again, this is only the core, not an application, so the controller just calls the model.</li> </ul> I found that a number of MVC systems confuse what should go into the Model and what should go into the Controller. I’m aiming for thin Controllers containing application logic, and thick Models containing the business logic. <h2 id="unit-testing">Unit Testing</h2> Backend-PHP used static global classes quite liberally, without even a thought around dependency injection. This makes unit testing unreliable and a real headache to add to the framework. Backend-Core, on the other hand, was designed to use <a href="http://martinfowler.com/articles/injection.html#ConstructorVersusSetterInjection">constructor injection</a> across the board, and includes a number of unit tests, supported by PHPUnit. This should increase the overall stability of the system. <h2 id="restful-service">RESTful Service</h2> Something I didn’t understand about REST when I initial wrote Backend-PHP, was that every REST URL should refer to a noun, with <a href="http://stackoverflow.com/questions/2001773/understanding-rest-verbs-error-codes-and-authentication/2022938#2022938">the only verbs being GET, POST, PUT and DELETE</a>, as specified by the HTTP request. This resulted in it not complying with REST standards, as a lot of the URLs contained extra verbs, or didn’t refer to nouns directly. Backend-Core supports the REST / HTTP verbs directly, and uses the same methods as CakePHP to identify which verb to use: <ul> <li>A <code class="highlighter-rouge">_method</code> POST variable</li> <li>A <code class="highlighter-rouge">X_HTTP_METHOD_OVERRIDE</code> header sent with the request</li> <li>The HTTP request’s method</li> </ul> This enables it to be used from a Web browser as well as from a REST compliant client. <h2 id="documentation-and-coding-standards">Documentation and Coding Standards</h2> The code complies with the <a href="http://framework.zend.com/manual/en/coding-standard.html">ZEND coding standard</a>, as enforced by <a href="http://pear.php.net/package/PHP_CodeSniffer/redirected">PHP Code Sniffer</a>. It’s documented using the <a href="http://www.phpdoc.org/">PHPdoc standard</a>. Hopefully this will make it easier for people to contribute to and extend the system. Feel free to fork, comment and use this. The Logstash Config Guide 2015-01-12T00:00:00+00:00 http://jrgns.net/blog/2015/01/12/logstash-config-guide The ELK stack has been becoming more and more popular, specifically <a href="http://www.elasticsearch.org/">Elasticsearch</a>. The <a href="http://www.google.com/trends/explore#q=elasticsearch">search volume</a> for it has been growing constantly since 2010. It’s faithful sidekick, <a href="http://logstash.net/">Logstash</a>, has been growing along with it, and ever since I’ve started looking at it, I’m just loving it more and more. The fact that you can string together totally disparate systems, from XMPP to Redmine and files on S3 to Nagios, with a simple config makes it a wonderful plaything and very useful utility. The rub, though, is in the config: Even though a great effort is made to write proper docs, I’ve found the documentation to be unclear, incomplete and sometimes confusing. So I’ve decided to do something about it: Pain Free Logstash Configuration <a href="https://leanpub.com/logstashconfigguide"> <img src="https://s3.amazonaws.com/titlepages.leanpub.com/logstashconfigguide/medium?1421001125" /> </a> Buy it now on <a href="https://leanpub.com/logstashconfigguide">Leanpub</a>  The plan is to write focused guides on some of the more popular Logstash plugins in the hopes of making it easier for people to set up and configure them. Each chapter will point out the bare minimum required to get you up and running and also highlight some of the more interesting, but not necessarily necessary settings for a plugin. You’ll also find a complete reference of all the available settings, with working examples for each of them. It’s still early days, but I’ll be publishing a number of preliminary posts on [EagerELK][5] to get things going. Subscribe there, or just buy the book on <a href="https://leanpub.com/logstashconfigguide">Leanpub</a>, to stay up to date. The book will be focusing on version 1.4.2 of Logstash (since 1.5 is still in beta), but I’ll update it once 1.5 is stable and being adopted. If you sign up for the 1.4.2 version, you’ll receive a free upgrade to the 1.5 version. Two books for the price of one! Any questions or comments are welcome, let me know below! Populate a Symfony 2 Form with the referring entity 2014-02-03T00:00:00+00:00 http://jrgns.net/blog/2014/02/03/populate-form-with-referring-entity-symfony In any web project it often happens that you have an entity, say a Group, to which you want to add a linked entity, say a Student. So you’ll have a “Add a Student” link on the page displaying the Group. When your users click through, there’s probably a dropdown with the different groups, and your users expect the Group they’re coming from to be prepopulated. There’s various ways to do this, but I’d like to show you a simple, non intrusive one for Symfony 2. You don’t have to add any parameters to the link, it just works. In your <code class="highlighter-rouge">Student</code> Controller, the <code class="highlighter-rouge">newAction</code> method: <figure class="highlight"><pre><code class="language-php" data-lang="php"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24</pre></td><td class="code"><pre><?php // src/My/Bundle/Controller/StudentController.php public function newAction() { $entity = new Student(); $form = $this->createForm(new StudentType(), $entity); // Get the refering URL Path $ref = str_replace("app_dev.php/", "", parse_url($request->headers->get('referer'),PHP_URL_PATH )); // Get the matching route $route = $this->container->get('router')->match($ref); if (empty($route['_route']) === false && $route['_route'] === 'group_show') { // Find the referring group if ($group = $this->get('doctrine')->getRepository('MyBundle:Group')->findOneById($route['id'])) { $form->get('group')->setData($group); } } return $this->render('MyBundle:Student:new.html.twig', array( 'entity' => $entity, 'form' => $form->createView(), )); } ?> </pre></td></tr></tbody></table></code></pre></figure> We basically try and match the referring URL to a route, and if found, retrieve that entity, and set it in the form. Simple! MS-SQL Stored Procedures in Sequel: Getting the value of output variables 2014-01-03T00:00:00+00:00 http://jrgns.net/blog/2014/01/03/ms-sql-stored-procs-with-output-variables-in-sequel We had the opportunity to muck around with a couple of technologies at <a href="http://www.tutuka.com">Tutuka</a> the last few weeks. I managed to do some work with Ruby, specifically <a href="http://www.sinatrarb.com/">Sinatra</a> and <a href="http://sequel.jeremyevans.net/">Sequel</a>. We use MS-SQL extensively and eventually we ran into an issue where we couldn’t get the values of output variables from stored procedures. After extensive googling we found out that it’s not supported. There were a couple of clues on how it might be done (particularly this <a href="https://github.com/rails-sqlserver/tiny_tds/issues/24">TinyTDS Issue</a>), so after chatting with Jeremy (the maintainer of Sequel) I decided to try and the necessary support. <h1 id="it-worked">It Worked!</h1> I was relatively surprised on how easy it was. As of yesterday (2014-01-02) it’s now part of the Sequel version 4.6 <div class="highlighter-rouge"><pre class="highlight"><code>Add Database#call_mssql_sproc on MSSQL for calling stored procedures and handling output parameters (jrgns, jeremyevans) (#748) </code></pre> </div> <h1 id="how-to-use-it">How to use it</h1> You can get a good idea on how to use it from the <a href="https://github.com/jeremyevans/sequel/blob/master/doc/mssql_stored_procedures.rdoc">documentation</a>, but here’s a summary. Let’s say your stored procedure is defined as follows: <div class="highlighter-rouge"><pre class="highlight"><code>CREATE PROCEDURE dbo.SequelTest( @Input varchar(25), @Output int OUTPUT ) AS SET @Output = LEN(@Input) RETURN 0 </code></pre> </div> If you don’t care about the type or the name of the output variable, execution is as simple specifying that an argument is an output parameter by passing the <code class="highlighter-rouge">:output</code> symbol as the argument value. <div class="highlighter-rouge"><pre class="highlight"><code>DB.call_mssql_sproc(:SequelTest, {:args => ['Input String', :output]}) > {:result => 0, :numrows => 1, :var1 => "1"} </code></pre> </div> The <code class="highlighter-rouge">result</code> and <code class="highlighter-rouge">numrows</code> element will contain the result code returned by the stored proc and the number of rows affected respectively. If you need to specify the type of the output variable, do so by specifying the second element of the array: <div class="highlighter-rouge"><pre class="highlight"><code>DB.call_mssql_sproc(:SequelTest, {:args => ['Input String', [:output, 'int']]}) > {:result => 0, :numrows => 1, :var1 => 1} </code></pre> </div> If you need to specify the name of the output variable, do so by specifying the third element of the array: <div class="highlighter-rouge"><pre class="highlight"><code>DB.call_mssql_sproc(:SequelTest, {:args => ['Input String', [:output, nil, 'output']]}) > {:result => 0, :numrows => 1, :output => "1"} </code></pre> </div> Enjoy! PHP: The slums of the Programming World? 2013-10-10T00:00:00+00:00 http://jrgns.net/blog/2013/10/10/is-php-the-slums-of-the-programming-world I presented this at <a href="http://www.tech4africa.com">Tech4Africa 2013</a>. The slides can be found <a href="http://jrgns.net/talks/phpslums">here</a>. <h1 id="quick-history-of-php">Quick History of PHP</h1> PHP was initially created by Rasmus Lerdorf when he got irritated by repetitive coding of CGI calls. He consolidated a number of functions into a C library and created a parser which would call these functions out of HTML. And so PHP was born! Versions 2 and 3 were formalizations and extensions of the libraries, still using Rasmus’ parser. The Zend Engine replaced Rasmus’ parser for version 4, introducing OOP capabilities, and in Version 5 Zend Engine 2 was introduced. Recent versions of PHP saw even more improvements, most notable namespaces in 5.3 and a built in web server and traits in 5.4 There’s no ideology or design philosophy, except that it should be a <a href="http://toys.lerdorf.com/archives/38-The-no-framework-PHP-MVC-framework.html">simple tool to solve problems</a>. <blockquote> “I’ve never thought of PHP as more than a simple tool to solve problems” <a href="https://twitter.com/rasmus/status/1938080214814720">Rasmus Lerdorf 2010</a> </blockquote> <h1 id="lies-damned-lies-and-">Lies, Damned Lies and …</h1> <img src="/img/tiobe-language-trends.png" alt="TIOBE Language trends" /> Despite a worsening reputation, the language hasn’t decreased in popularity. <img src="/img/web-language-trends.png" alt="Web Language Trends" /> Less searches are being done for PHP. <img src="/img/framework-trends.png" alt="Framework Trends" /> If you look at the JavaScript vs JQuery trends, JavaScript is also decreasing, but JQuery is increasing. It seems like the programming community is realising that frameworks are a good idea. <img src="/img/php-framework-trends.png" alt="PHP Framework Trends" /> So if you look at PHP frameworks, searches on them are also increasing, especially after 2013 when Zend 2 and Symfony 2 were released. <img src="/img/php-usage.png" alt="PHP Usage" /> No shortage of hosts offering PHP. <h1 id="who-uses-it">Who uses it?</h1> <ul> <li>User facing portion of Facebook through HipHop PHP (which generates C++)</li> <li>Wordpress - Application turned framework</li> <li>Yahoo (for now). They are replacing it with Node.</li> <li>Wikipedia / Mediawiki</li> </ul> <h1 id="embrace-the-cactus">Embrace the Cactus</h1> It’s easy to create insecure / buggy code in PHP: <div class="language-php highlighter-rouge"><pre class="highlight"><code> mysql_query("SELECT * FROM `users` WHERE `username` = $_GET['username']"); </code></pre> </div> PHP accounts for only 18% of applications covered by the <abbr class="Common Vulnerabilities and Exposures"><a href="http://www.coelho.net/php_cve.html">CVE</a></abbr>, but accounts for almost 29% of the vulnerabilities It’s developed a bad reputation in the programming community: <blockquote> Bringing a knife to a gun fight is like bringing PHP to a modern web stack. <a href="https://twitter.com/iamdevloper/status/380285502580338688">@iamdevloper</a> </blockquote> Settings such as <code class="highlighter-rouge">register_globals</code> and <code class="highlighter-rouge">magic_quotes</code> that defaulted to <code class="highlighter-rouge">on</code> opened up a lot of security holes No unified community. In general, the community is great. In the rare cases where the PHP documentation (or even the language) is insufficient, you can usually find something of value in the comments. Yet, there is a lot of conflict in both the official (like PHP internals) and the unofficial (like <a href="http://www.php-fig.org/">PHP-FIG</a>) communities. If you look at the debate on what to use for indentation, <a href="http://philsturgeon.co.uk/blog/2013/04/psr2-the-tough-decision">tabs</a> vs <a href="https://github.com/php-fig/fig-standards/pull/35">spaces</a>, the community is in total disarray. <img src="/img/tabs-not-spaces.png" alt="Tabs Not Spaces Slugfest" /> Inconsistencies in naming functions: <div class="language-php highlighter-rouge"><pre class="highlight"><code>strrev // String Function str_repeat // String Function explode // String Function // At least somewhat consistent strpos($haystack, $needle) array_key_exists($needle, $haystack) </code></pre> </div> Object Oriented, or just Object Capable? - <a href="http://michaelkimsal.com/blog/php-is-not-object-oriented/">Michael Kimsal</a> So you can do Object Oriented Programming as well as Procedural and Functional Programming in PHP (especially PHP 5.3 and later). <h1 id="see-you-on-the-flip-side">See you on the flip side…</h1> <h2 id="php-53">PHP 5.3</h2> PHP 5.3 is for me a watershed version on PHP’s history - It introduced a number of features / capabilities that made it possible for some of the other factors I’ll mention to come into existence. The most notable feature is namespaces, making it easier to name classes in a sane way. A lot of people welcomed the introduction of lambda functions and closures. I think the PHP developers and community realized that PHP was becoming outdated and had a lot of valid issues that needed to be addressed. For me 5.3 was a definite step in the right direction. It introduced (amongst other things): <ul> <li>Namespaces</li> <li>Closures and Lambda functions</li> </ul> <h2 id="php-fig">PHP-FIG</h2> Although it seems to generate a lot of conflict, the PHP-FIG has managed to at least get the different frameworks talking. It’s an unofficial (as in not sponsored by PHP) group formed by various frameworks and projects whose aim it is to get the projects talking and to find ways to work together. It’s not always plain sailing, but it seems to get to valuable decisions. Eventually. <ul> <li><a href="https://github.com/php-fig/fig-standards/blob/master/accepted/PSR-0.md">PSR-0</a> has enabled the creation of packaging software such as composer / packagist</li> <li><a href="https://github.com/php-fig/fig-standards/blob/master/accepted/PSR-1-basic-coding-standard.md">PSR-1</a> and <a href="https://github.com/php-fig/fig-standards/blob/master/accepted/PSR-2-coding-style-guide.md">PSR-2</a> generated a lot of conflict, but at least all of the participating frameworks partially agreed on coding standards</li> <li><a href="https://github.com/php-fig/fig-standards/blob/master/accepted/PSR-3-logger-interface.md">PSR-3</a> and <a href="https://github.com/php-fig/fig-standards/blob/master/proposed/psr-4-autoloader/psr-4-autoloader.md">PSR-4</a> aren’t momentous decisions, but they at least pave the way forward for increased collaboration</li> </ul> <h2 id="packagist--composer">Packagist / Composer</h2> <a href="http://getcomposer.org/">Composer</a> is a tool to manage dependencies for a project, and manages and installs dependencies on a per project basis. Very much on par with node’s npm and ruby’s bundler. <a href="https://packagist.org/">Packagist</a> is a repository for composer packages and makes it dead simple to find a package / library that will address your specific need. Before this, we had PEAR, but that was usually a headache to set up and version properly locally, not to mention on a shared host. Now it’s as simple as doing <div class="language-bash highlighter-rouge"><pre class="highlight"><code> composer require "vendor/library" </code></pre> </div> To a certain extent the improvements in PHP 5.3 and some of the work from PHP-FIG made this possible, although composer could have existed without them. <h2 id="modern-frameworks">Modern Frameworks</h2> The combination of PHP 5.3 (specifically namespaces and proper OO support), the work of the PHP-GIH and the development of Composer made it possible to not only create frameworks such as Symfony2, Zend2 and Laravel, but also to reuse components from these frameworks in other projects. Drupal 8 uses a lot of Symfony2’s components. PPI is a framework built using both Zend2 and Symfony2 components. You can now easily choose and combine libraries without having to manage class names or dependencies. This has at least two direct advantages for project developers: <ol> <li>Instant access to a whole range of precoded, quality, libraries.</li> <li>No more wasting time on managing dependencies and name clashes.</li> </ol> <h1 id="so-slum-or-diamond-in-the-rough">So, Slum or Diamond in the Rough?</h1> <blockquote> Good systems cater for users’ mistakes, great systems also cater for developers’ mistakes… <a href="https://twitter.com/jrgns/status/365357629469499392">Jrgns</a> </blockquote> My opinion is, there is no right answer. To a certain extent, PHP can be described as a horrible language that makes it very easy to introduce bugs and security issues. It doesn’t give you guidance on how to do anything, and can’t seem to decide if it’s procedural or object oriented. In the same breath, it gives you a lot of freedom to code according to your style, and makes a lot of common coding tasks very easy to do. Successful business have been built and are still running on PHP. If you’ve never coded anything but PHP and have no formal programming education (including good programming books), you’re probably coding crappy PHP. Probably. I’d suggest you start reading up on good programming practices, start reading code that’s of a high quality, and start learning other programming languages, especially stricter ones like Java, or start using frameworks such as Symfony2 and Laravel that at least guides you in the right direction. Speaking multiple natural languages keeps your brain sharp and young. Coding in <a href="http://www.sciencemag.org/content/330/6002/332.summary">multiple languages</a> makes you a better coder. If you know what you’re doing and you perhaps have some extra experience in a more structured language such as Java or Ruby, you’re probably coding quality PHP. It’s un-opinionated, so, as long as you know what you’re doing, it can be a great tool. Don’t discount the language just because it’s lost it’s coolness factor. You can write quality code in PHP. <img src="/img/sheer-iron-willed-discipline.png" alt="Sheer Iron WIlled Discipline" /> <blockquote> If you can pull it off, you’re badass as hell, but most of the time you’re only going to end up stabbing yourself. <a href="https://twitter.com/ohpalmer/status/380290678502461440">ohpalmer</a> </blockquote> A DIY Dynamic DNS service using Cloudflare 2013-09-26T00:00:00+00:00 http://jrgns.net/blog/2013/09/26/diy-dynamic-dns-using-cloudflare DynDNS was always the dynamic IP service of choice, but I could never get it to work properly, and I was always stuck with their lame domain names as I’m too cheap to pay for their serivces. Luckily services of all kinds, including DNS Providers, have caught on to the fact that people want API’s. If there’s an API to do DNS updates, I can script a solution. Some of my domains run off of <a href="https://www.cloudflare.com/">Cloudflare</a> which needs to overwrite your DNS. And, hey presto, they have an <a href="http://www.cloudflare.com/docs/client-api.html">API</a>. So here it is: <figure class="highlight"><pre><code class="language-bash" data-lang="bash"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 5 6 7 8 9 10 11 12</pre></td><td class="code"><pre>#!/bin/bash curl https://www.cloudflare.com/api_json.html \ -d 'a=rec_edit' \ -d 'tkn=1234512345qweqwe1234512345' \ -d 'id=12345' \ -d 'email=jrgns@jrgns.net' \ -d 'z=jrgns.net' \ -d 'type=A' \ -d 'name=home' \ -d "content=$1" \ -d 'service_mode=0' \ -d 'ttl=3600' </pre></td></tr></tbody></table></code></pre></figure> Quick and dirty. This updates a domain (home.jrgns.net) with id 12345 with the IP address that’s passed as the first argument to the script. There’s a bunch of ways to get your external IP, my favourite is <code class="highlighter-rouge">curl http://api.externalip.net/ip</code>, but I just run this script as the <code class="highlighter-rouge">postscript</code> of my ddclient (which updates another service) <figure class="highlight"><pre><code class="language-text" data-lang="text"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 5 6 7 8 9 10</pre></td><td class="code"><pre># /etc/ddclient.conf protocol=dyndns2 use=web, web=myip.dnsomatic.com ssl=yes server=updates.someservice.com login=mylogin@gmail.com password='123456' daemon=3600 postscript=/home/jrgns/bin/cloudflare_dynamic_dns.sh </pre></td></tr></tbody></table></code></pre></figure> I’m sure the same can be done with outer services such as Route 53? Has anyone tried something like this? Setting a default order for Doctrine 2 / Symfony 2 2013-09-17T00:00:00+00:00 http://jrgns.net/blog/2013/09/17/setting-a-default-order-for-doctrine-symfony-2 Usually when you’re working with data you expect some kind of default ordering. It might be the primary key, or, more commonly, the data set will be ordered by a timestamp field or the data’s most important property. In Doctrine you can create <a href="http://symfony.com/doc/current/book/doctrine.html#custom-repository-classes">custom repository classes</a> to tweak queries to your liking: <figure class="highlight"><pre><code class="language-php" data-lang="php"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18</pre></td><td class="code"><pre><?php // src/Acme/StoreBundle/Entity/ProductRepository.php namespace Acme\StoreBundle\Entity; use Doctrine\ORM\EntityRepository; class ProductRepository extends EntityRepository { public function findAllOrderedByName() { return $this->getEntityManager() ->createQuery( 'SELECT p FROM AcmeStoreBundle:Product p ORDER BY p.name ASC' ) ->getResult(); } } ?> </pre></td></tr></tbody></table></code></pre></figure> This is all fine and dandy, but do you really want to repeat yourself everytime you write a query for a model that will almost always be ordered by the same field? Or get caught out by calls to default repository methods that don’t use the ordering you like? No, you don’t. Enter this piece of code: <figure class="highlight"><pre><code class="language-php" data-lang="php"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21</pre></td><td class="code"><pre><?php // src/Acme/StoreBundle/Entity/ProductRepository.php namespace Acme\StoreBundle\Entity; use Doctrine\ORM\EntityRepository; class ProductRepository extends EntityRepository { public function findBy(array $criteria, array $orderBy = null, $limit = null, $offset = null) { // Default Ordering $orderBy = $orderBy === null ? array('added' => 'desc') : $orderBy; // Default Filtering if (array_key_exists('active', $criteria) === false) { $criteria['active'] = 1; } return parent::findBy($criteria, $orderBy, $limit, $offset); } } ?> </pre></td></tr></tbody></table></code></pre></figure> The <a href="http://www.doctrine-project.org/api/orm/2.3/class-Doctrine.ORM.EntityRepository.html#_findBy">findBy</a> method of the EntityRepository class is the method used by most (if not all) of the the repository methods that fetch data. This includes <code class="highlighter-rouge">findBy</code> methods where you can specify an arbitrary column and value: <figure class="highlight"><pre><code class="language-php" data-lang="php"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4</pre></td><td class="code"><pre><?php // find a group of products based on an arbitrary column value $products = $repository->findByPrice(19.99); ?> </pre></td></tr></tbody></table></code></pre></figure> So to ensure a default ordering, we override this method and check if an orderBy clause has been set. If not, set it to our default ordering — in this case by the <code class="highlighter-rouge">added</code> field, descending. For good measure we also add default filtering — the <code class="highlighter-rouge">active</code> field needs to equal 1. Once that’s done, call the parent method with the default values set. Easy as π Pre-populating an Entity in a Symfony 2 Form 2013-07-22T00:00:00+00:00 http://jrgns.net/blog/2013/07/22/prepopulating-entity-in-symfony-2-form I’ve been working on <a href="http://www.teach-a-rific.com">teach-a-rific.com</a> in my spare time, and I’m learning a lot about how <a href="http://symfony.com/doc/current/book/forms.html">Symfony 2 Forms</a> work. One thing I need to do a lot, is to show a form with a <a href="http://symfony.com/doc/current/reference/forms/types/entity.html">entity field</a> pre-populated. In this situation you also don’t want the user to be able to edit the field. This is how I did it. First, you need to retrieve the mapped entity, in this case a <code class="highlighter-rouge">Student</code>, and set it in the entity which you’re creating a form for. In this case a <code class="highlighter-rouge">ContactDetail</code>. This all happens in the <code class="highlighter-rouge">newAction</code> method of the <code class="highlighter-rouge">ContactDetailController</code>. Notice how I save the student ID in a session variable. More on this later. <figure class="highlight"><pre><code class="language-php" data-lang="php"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17</pre></td><td class="code"><pre><?php //ContactDetailController::newAction $detail = new ContactDetail(); if ($studentId = $request->query->get('student_id')) { $student = $this ->getDoctrine() ->getManager() ->getRepository('NotesNoteBundle:Student') ->find($studentId); if ($student) { $this->get('session')->set('contact_detail:create:student', $studentId); $detail->setStudent($student); } } $form = $this->createForm(new ContactDetailType(), $detail); </pre></td></tr></tbody></table></code></pre></figure> On to the form. Quite simply, if the entity already has a <code class="highlighter-rouge">Student</code>, don’t add the field for the student. <figure class="highlight"><pre><code class="language-php" data-lang="php"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18</pre></td><td class="code"><pre><?php use Symfony\Component\Form\AbstractType; use Symfony\Component\Form\FormBuilderInterface; use Symfony\Component\OptionsResolver\OptionsResolverInterface; class ContactDetailType extends AbstractType { public function buildForm(FormBuilderInterface $builder, array $options) { if ($builder->getData()->getStudent() === null) { $builder->add('student'); } $builder ->add('type', 'choice', array('choices' => array('Email' => 'Email', 'Cellphone' => 'Cellphone'))) ->add('content') ; } } </pre></td></tr></tbody></table></code></pre></figure> On to the actual form. If the student field is defined in the form, display the form widget. Otherwise display the student in an uneditable text input. Displaying the student can be skipped if it’s not necessary. <figure class="highlight"><pre><code class="language-php" data-lang="php"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 5 6 7 8 9 10</pre></td><td class="code"><pre>{% if form.student is defined %} {{ form_row(form.student, { form_type: 'horizontal' }) }} {% elseif form.vars.data.student.id is defined %} <div class="control-group"> <label class="control-label">Student</label> <div class="controls"> <span class="input uneditable-input">{{ form.vars.data.student }} </div> </div> {% endif %} </pre></td></tr></tbody></table></code></pre></figure> And on to the <code class="highlighter-rouge">updateAction</code>. Just get the <code class="highlighter-rouge">Student</code> whose ID we stored in the session, and set it on the <code class="highlighter-rouge">ContactDetail</code> before populating the form: <figure class="highlight"><pre><code class="language-php" data-lang="php"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 5 6 7 8 9 10 11 12 13 14</pre></td><td class="code"><pre><?php //ContactDetailController::createAction $entity = new ContactDetail(); if ($studentId = $this->get('session')->get('contact_detail:create:student')) { $student = $this ->getDoctrine() ->getManager() ->getRepository('NotesNoteBundle:Student') ->find($studentId); if ($student) { $entity->setStudent($student); } } </pre></td></tr></tbody></table></code></pre></figure> Simple enough. Any suggestions on how to improve it, or comments on how to do it differently? A Summary of Symfony 2 Authorization 2013-04-29T00:00:00+00:00 http://jrgns.net/blog/2013/04/29/summary-of-symfony-2-authorization Symfony 2 comes with a great <a href="http://symfony.com/doc/current/book/security.html">security component</a> that provides both <a href="http://symfony.com/doc/current/components/security/authentication.html">Authentication</a> (identifying the user) and <a href="http://symfony.com/doc/current/components/security/authorization.html">Authorization</a> (determining the permissions for the user) functionality. It has sane defaults, works out of the box, and is extendable enough to cater for any scenario. This post describe the default authorization functionality available to the developer. Out of the box, Symfony 2 offers two method with which to authorize a user: <ul> <li>Role based authorization</li> <li>Access control lists (ACL’s)</li> </ul> <h2 id="role-based-authorization">Role Based Authorization</h2> With role based authorization, you specify a number of roles (which can be hierarchical) and specify the access of these roles, all within your application config: <div class="highlighter-rouge"><pre class="highlight"><code>#project/app/config/security.yml security: role_hierarchy: ROLE_ANONYMOUS: ROLE_USER: ROLE_ANONYMOUS ROLE_ADMIN: ROLE_USER ROLE_SUPER_ADMIN: [ROLE_ADMIN, ROLE_ALLOWED_TO_SWITCH] access_control: - { path: ^/login, roles: IS_AUTHENTICATED_ANONYMOUSLY } - { path: ^/public, roles: IS_AUTHENTICATED_ANONYMOUSLY } - { path: ^/, roles: ROLE_USER } - { path: ^/admin/, role: ROLE_ADMIN } </code></pre> </div> In the config above, there’s roles (or types of users). Anonymous, normal users, admin users and super admin users. Anonymous users can only access the public part of the site. Logged in users can access the whole site, except for the admin area. Admin users can access the admin area of the site. Relatively simple and easy to set up, but not very customizable on a fine grained level. <h2 id="access-control-lists">Access Control Lists</h2> Say for instance that users generate content, and that users can only access the content they generated. The role based rules are too general for this, so we’ll turn to Access Control Lists for a solution. The concept of ACL’s boils down to a list of many to many relations between users and resources defining who has access to what. Even though Symfony provides built in <a href="http://symfony.com/doc/current/cookbook/security/acl.html">ACL functionality</a>, this can become hairy quite quickly, as you need to manage Access Control Entries (ACE) in your controller code. Although Symfony provides an <a href="http://api.symfony.com/2.0/Symfony/Component/Security/Acl/Dbal/AclProvider.html">ACL Provider</a> with which the coder can create and check ACL’s, it can become hairy quite quickly, as it needs to be managed in each and every Controller Action. This boilerplate code wil appear in controller actions that deal with creating Entities and their ACL’s: <figure class="highlight"><pre><code class="language-php" data-lang="php"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20</pre></td><td class="code"><pre><?php use Symfony\Component\Security\Acl\Domain\ObjectIdentity; use Symfony\Component\Security\Acl\Domain\UserSecurityIdentity; use Symfony\Component\Security\Acl\Permission\MaskBuilder; // Inside a controller action // creating the ACL $aclProvider = $this->get('security.acl.provider'); $objectIdentity = ObjectIdentity::fromDomainObject($entity); $acl = $aclProvider->createAcl($objectIdentity); // retrieving the security identity of the currently logged-in user $securityContext = $this->get('security.context'); $user = $securityContext->getToken()->getUser(); $securityIdentity = UserSecurityIdentity::fromAccount($user); // grant owner access $acl->insertObjectAce($securityIdentity, MaskBuilder::MASK_OWNER); $aclProvider->updateAcl($acl); </pre></td></tr></tbody></table></code></pre></figure> The “creating the ACL” block retrieves the Security Provider, generates an identity for the object using the <code class="highlighter-rouge">ObjectIdentity</code> class, and then creates the ACL using the security provider. The second block retrieves the Security Context and the current user. From there it generates the an identifier for the user using the <code class="highlighter-rouge">UserSecurityIdentity</code> class. The third block then inserts an Access Control Entry and updates the ACL. That’s a lot of code. If there’s a generic way to handle this, please let me know in the comments. <h2 id="behind-the-scenes">Behind the Scenes</h2> The key component in how Symfony manages authentication is the <a href="http://api.symfony.com/2.2/Symfony/Component/Security/Core/Authorization/AccessDecisionManager.html">AccessDecisionManager</a>. Each time a request is made, the <code class="highlighter-rouge">AccessDecisionManager</code> lets registered Voters vote on if the user may access the resource. The voters will determine their vote by inspecting the current security token, a set of attributes, and an optional object. Access will then be granted (or denied) depending on the votes and the <code class="highlighter-rouge">AccessDecisionManager's</code> decision strategy. Both the role based auth and ACL’s use this mechanism. I’ll go into how to use the <code class="highlighter-rouge">AccessDecisionManager</code> and Voters in a later post. I'm the most Organized Disorganized person you'll meet 2013-02-28T00:00:00+00:00 http://jrgns.net/blog/2013/02/28/im-the-most-organized-disorganized-person-youll-meet One of the most telling compliments I ever received was from a good friend: “You’re the most organized, disorganized person I know!”. It’s quite telling once you wrap your head around it. It stuck by me, and it’s helping me shape my career. I’ve always been absent minded. Classic “if you’re head wasn’t tied to your body, you’ll loose it!” stuff. I’d put something down, and then search for it for half an hour, only to discover it’s been in my pocket the whole time. One time I was talking to my mom on the phone, realised that I didn’t know where my phone was, and proceed to tell my mom about it… At the end of the day, details just didn’t stick in my head. Now I’m in an industry where detail is everything. Businesses lose thousands because of a missing negative sign. One line of code out of place crashes an application and loses a company users, money and its reputation. I always tell people that creating systems is like eating an elephant: bit by bit. what they don’t realise is that if you get one byte wrong, everything comes crashing down. So how wise is it for a guy who can’t keep track of both his sandles to be in the details business? Either very foolish, or a stroke of genius. Genius? Let’s look that compliment again: The most organized disorganized person. To cope with all these details, I started to rely on systems. I organized my life in such a way that I didn’t need to worry about the details. Using systems ensures that even the most unorganized person won’t miss any of the details. I used calandering to keep track of appointments. A todo list for tasks. Evernote for easy to forget yet important info. When I go out the door I chant “Phone, keys, wallet” to ensure that I have the essentials. Every day I’m hacking my brain. The take away is that while people make mistakes, systems rarely do. Don’t rely on people to take care of the details, rather put systems in place to take care it. If it’s tricky or complicated to deploy an app, script it. If a process is long and protracted, automate it. Create unit and behaviour tests for your code. Add state checks to the application. Run regular audits on the numbers and make sure they balance. In the age of rapid development and continuous deployment, just writing your app isn’t enough anymore. You need to write systems around your app to ensure that it is coded correctly, deployed correctly, working correctly. PHP's array_merge_recursive function explained 2013-01-08T00:00:00+00:00 http://jrgns.net/blog/2013/01/08/php-array-merge-recursive-function-explained Reading the PHP docs, it’s not immediately clear how the <a href="http://php.net/manual/en/function.array-merge-recursive.php"><code class="highlighter-rouge">array_merge_recursive</code></a> function behaves. I’ve setup a couple of test cases and compared them with the plain <a href="http://php.net/manual/en/function.array-merge.php"><code class="highlighter-rouge">array_merge</code></a> function to explain it further: The following snippet set’s up a couple of arrays, and outputs them for completeness sake. <figure class="highlight"><pre><code class="language-php" data-lang="php"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 5 6 7 8 9 10 11 12 13 14</pre></td><td class="code"><pre><?php $cats = array( 'white' => array('Kitty'), 'brown' => 'Bach', 'mixed' => array('Tomcat') ); $dogs = array( 'white' => 'Bowser', 'brown' => 'Rex', 'mixed' => ('Mutt') ); var_dump('Cats', $cats, 'Dogs', $dogs); ?> </pre></td></tr></tbody></table></code></pre></figure> <pre>string 'Cats' (length=4) array 'white' => array 0 => string 'Kitty' (length=5) 'brown' => string 'Bach' (length=4) 'mixed' => array 0 => string 'Tomcat' (length=6) string 'Dogs' (length=4) array 'white' => string 'Bowser' (length=6) 'brown' => string 'Rex' (length=3) 'mixed' => string 'Mutt' (length=4) </pre> This snippet highlights what happens when the arrays being merged have matching keys. <figure class="highlight"><pre><code class="language-php" data-lang="php"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4</pre></td><td class="code"><pre><?php var_dump('Cats and Dogs - plain', array_merge($cats, $dogs)); var_dump('Cats and Dogs - recursive', array_merge_recursive($cats, $dogs)); ?> </pre></td></tr></tbody></table></code></pre></figure> All values that’s not already in an array will be placed in an array, and merged with other values with the same keys. Arrays are left as they are and then merged. <pre> array 'white' => array 0 => string 'Kitty' (length=5) 1 => string 'Bowser' (length=6) 'brown' => array 0 => string 'Bach' (length=4) 1 => string 'Rex' (length=3) 'mixed' => array 0 => string 'Tomcat' (length=6) 1 => string 'Mutt' (length=4) </pre> Compare this with the normal <code class="highlighter-rouge">array_merge</code> function, where the values of the latter arrays replace all the values with the same key. <pre> array 'white' => string 'Bowser' (length=6) 'brown' => string 'Rex' (length=3) 'mixed' => string 'Mutt' (length=4) </pre> Values with numerical keys get added to the original array with a renumbering of keys. This is the same behaviour as that of <code class="highlighter-rouge">array_merge</code>. Setting up the Swiftmailer Spooler 2012-12-20T00:00:00+00:00 http://jrgns.net/blog/2012/12/20/setting-up-the-swiftmailer-spooler The <a href="http://swiftmailer.org/">Swiftmailer</a> library comes with a nifty feature where mails can be spooled to disk and sent later asynchronously instead of mailing them immediately. This results in quicker response times from your scripts, and by extension happier users. This post will explain how to set it up outside of the Symfony 2 framework. <h2 id="spooling-the-mail">Spooling the Mail</h2> Swiftmailer defines a number of Transports with which mails can be sent. A common one is the <code class="highlighter-rouge">Swift_SmtpTransport</code>. To spool mails, we’ll use the <code class="highlighter-rouge">Swift_SpoolTransport</code> for the initial sending, and then still use <code class="highlighter-rouge">Swift_SmtpTransport</code> for the background sending. The following comes (with some changes) from a post in the <a href="https://groups.google.com/forum/?fromgroups=#!searchin/swiftmailer/spool/swiftmailer/xQPP5qtNnMA/k3QXcRebG-wJ">Swiftmailer Google Group</a>: <figure class="highlight"><pre><code class="language-php" data-lang="php"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17</pre></td><td class="code"><pre><?php // Setup the spooler, passing it the name of the folder to spool to $spool = new Swift_FileSpool(__DIR__ . "/spool"); // Setup the transport and mailer $transport = Swift_SpoolTransport::newInstance($spool); $mailer = Swift_Mailer::newInstance($transport); // Create a message $message = Swift_Message::newInstance('Excellent Subject') ->setFrom(array('sende...@domain.com' => 'John Doe')) ->setTo(array('your_emailAddress@domain.com')) ->setBody('spool messages !!!'); // Send the message $result = $mailer->send($message); echo "SPOOLED $result emails"; </pre></td></tr></tbody></table></code></pre></figure> This will spool the email to specified folder instead of sending it. At this stage, the actual sending of the mail hasn’t happened yet. You will need to flush the spooled messages using a cron or background script that calls the flashQueue method to do that. <h2 id="sending-the-mail">Sending the Mail</h2> The background script will use the <code class="highlighter-rouge">Swift_SmtpTransport</code> to send the spooled mails: <figure class="highlight"><pre><code class="language-php" data-lang="php"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23</pre></td><td class="code"><pre><?php //create an instance of the spool object pointing to the right position in the filesystem $spool = new Swift_FileSpool(__DIR__."/spool"); //create a new instance of Swift_SpoolTransport that accept an argument as Swift_FileSpool $transport = Swift_SpoolTransport::newInstance($spool); //now create an instance of the transport you usually use with swiftmailer //to send real-time email $realTransport = Swift_SmtpTransport::newInstance( "smtp.gmail.com", "465", "ssl" ) ->setUsername("username") ->setPassword("password"); $spool = $transport->getSpool(); $spool->setMessageLimit(10); $spool->setTimeLimit(100); $sent = $spool->flushQueue($realTransport); echo "SENT $result emails"; </pre></td></tr></tbody></table></code></pre></figure> And that’s it. All that’s left now is to call the background script periodically using something like cron. Building PHP Projects with Jenkins 2012-06-09T18:46:23+00:00 http://jrgns.net/building-php-projects-with-jenkins/building-php-projects-with-jenkins It can become quite difficult to keep PHP code structured and clean. There’s a couple of good tools out there (<a href="http://pear.php.net/package/PHP_CodeSniffer/redirected">PHP Code Sniffer</a>, <a href="http://phpmd.org/">PHP Mess Detector</a>, etc) to help you manage it, but it adds a lot of overhead that can bring your productivity down. One way to keep the advantage of checking your code without taking a productivity hit, is using a build system like Jenkins. Look at <a href="http://erichogue.ca/2011/05/php/continuous-integration-in-php/">this blog post</a> to setup Jenkins and the various tools. There’s also a <a href="http://jenkins-php.org/">Jenkins PHP Template</a> that will help a great deal in getting things started. I’m planning to play around with it, and post updates here about how it goes. Decorator Pattern Implemented Properly In PHP 2012-03-17T11:14:56+00:00 http://jrgns.net/decorator-pattern-implemented-properly-in-php/decorator-pattern-implemented-properly-in-php While working on <a href="http://backend-php.net">Backend-PHP</a> I needed to do a proper implementation of the <a href="http://en.wikipedia.org/wiki/Decorator_pattern">Decorator Pattern</a> in PHP. Just googling PHP decorator pattern will come up with a number of simple solutions, but none of them are usable in a general, robust way that a framework requires. So I extended and tweaked the implementations a bit. ##TL;DR Most decorator pattern implementations for PHP found on the web is broken. They need some extra functionality to work properly, especially for nested decorators. See the complete solution in the <a href="https://github.com/backend/Backend-PHP-Core/blob/master/Decorators/Decorator.php">Backend-PHP code</a> ##The Basics I’m assuming you know how decorators work. If not, read up on it first! I’ve implemented a base <code class="highlighter-rouge">Decorator</code> class to handle the base functionality for Decorators. The simple implementations out there serve as a good base for an initial implementation: <div class="highlighter-rouge"><pre class="highlight"><code>class Decorator { protected $object; //The object to decorate function __construct( $object) { $this->object = $object; } } </code></pre> </div> This is enough to get you started, but you will need to wrap the original object’s properties and methods manually. Let’s define a couple of magic functions to do that automatically: <div class="highlighter-rouge"><pre class="highlight"><code>public function __call($method, $args) { if (is_callable($this->object, $method) { return call_user_func_array(array($this->object, $method), $args); } throw new Exception( 'Undefined method - ' . get_class($this->object) . '::' . $method ); } public function __get($property) { if (property_exists($this->object, $property)) { return $this->object->$property; } return null; } public function __set($property, $value) { $this->object->$property = $value; return $this; } </code></pre> </div> That should give you a working basic implementation. ##More Advanced That will work as long as you don’t nest decorators, ie, decorate an already decorated object: <div class="highlighter-rouge"><pre class="highlight"><code>$object = new DecoratorOne(new DecoratorTwo(new SomeClass())); </code></pre> </div> Why? The magic functions will check the decorated object for the properties and methods, but in the case of <code class="highlighter-rouge">DecoratorOne</code> above, the decorated object will be another decorator, and you won’t get the expected behaviour. This happens because <code class="highlighter-rouge">property_exists</code> doesn’t trigger the <code class="highlighter-rouge">__get</code> (or <code class="highlighter-rouge">__set</code>) methods. <code class="highlighter-rouge">is_callable</code> does check for the <code class="highlighter-rouge">__call</code> function, but that doesn’t necessarily produce the expected results, especially if you stack decorators that modify method execution. We solve this by adding two more functions: ###getOriginalObject <div class="highlighter-rouge"><pre class="highlight"><code>public function getOriginalObject() { $object = $this->object; while ($object instanceof Decorator) { $object = $object->getOriginalObject(); } return $object; } </code></pre> </div> <code class="highlighter-rouge">getOriginalObject</code> will return the original, undecorated object. This is handy to get the correct class name or access to undecorated methods and properties: <div class="highlighter-rouge"><pre class="highlight"><code>echo get_class($decoratedClass) //Will give the name of the Decorator echo get_class($decoratedClass->getOriginalObject()) //Will give the name of the original object </code></pre> </div> ###isCallable <div class="highlighter-rouge"><pre class="highlight"><code>public function isCallable($method, $checkSelf = false) { //Check the original object $object = $this->getOriginalObject(); if (is_callable(array($object, $method))) { return $object; } //Check Decorators $object = $checkSelf ? $this : $this->object; while ($object instanceof Decorator) { if (is_callable(array($object, $method))) { return $object; } $object = $this->object; } return false; } </code></pre> </div> <code class="highlighter-rouge">isCallable</code> will check the original object for the specified method, and if not found, it will check the other decorators for the method. The option exists to check itself as well. ##The Result Now we can modify the magic functions like so: <div class="highlighter-rouge"><pre class="highlight"><code>public function __call($method, $args) { if ($object = $this->isCallable($method)) { return call_user_func_array(array($object, $method), $args); } throw new Exception( 'Undefined method - ' . get_class($this->getOriginalObject()) . '::' . $method ); } public function __get($property) { $object = $this->getOriginalObject(); if (property_exists($object, $property)) { return $object->$property; } return null; } public function __set($property, $value) { $object = $this->getOriginalObject(); $object->$property = $value; return $this; } </code></pre> </div> This will give you a robust Decorator Pattern implementation from which you can then create more Decorators. See the complete implementation in the <a href="https://github.com/backend/Backend-PHP-Core/blob/master/Decorators/Decorator.php">Backend-PHP code</a> The Low Hanging Fruit Of Testing 2012-03-16T15:14:03+00:00 http://jrgns.net/the-low-hanging-fruit-of-testing/the-low-hanging-fruit-of-testing I’ve been getting more into Testing (specifically Unit Testing) the past few weeks, and although I’m still struggling to firstly get it right, and secondly see the long term benefits, I know that the quality of my code is already better. Let’s call it the low hanging fruit of Testing. I find that I’m reordering and sometimes rewriting functions, and sometimes whole classes, because just the need to test it already shows some design flaws. A class with too many private methods, a function with no clear input / output, all of that shows up when you start to write tests. The fact that I have to think of my code and classes in terms of an API requires that I make sure of what the purpose of a function is, and what it’s signature and return should look like. Because it’s difficult to test large, complex functions, I’m refactoring those into smaller, more modular functions. So I’m spending more time with the code, making improvements as I read and think about the code. So yeah, I’m not utilizing the full power of testing yet (not to speak of Test Driven Development), but I’m already enjoying the fruits. json-encode'ing Private and Protected Properties 2012-02-24T06:51:15+00:00 http://jrgns.net/json-encode-ing-private-and-protected-properties/json-encode-ing-private-and-protected-properties When you <code class="highlighter-rouge">json_encode</code> an object, it will ignore all protected and private properties if the <code class="highlighter-rouge">json_encode</code> isn’t called from within the object. Sometimes you want to output the protected and private properties of an object when you’re JSON encoding it. Here’s how to do it in an easy and elegant way: ##The Problem The properties of an object <code class="highlighter-rouge">json_encode</code> includes in it’s eventual output depends on the scope of the function call: If it’s from inside the object, it will include all the properties, private and protected. If it’s from outside of the object, it will only include the public properties. <figure class="highlight"><pre><code class="language-php" data-lang="php"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 5 6 7 8 9 10 11 12 13 14</pre></td><td class="code"><pre><?php class Tester { public $public = 'Public'; protected $protected = 'Protected'; private $private = 'Private'; function foo() { echo json_encode($this); } } $test = new Tester(); $test->foo(); //Outputs {"public":"Public","protected":"Protected","private":"Private"} echo json_encode($test); //Outputs {"public":"Public"} </pre></td></tr></tbody></table></code></pre></figure> This isn’t necessarily the desired outcome. Sometimes you want more control over what is being included in the encoding function. ##The Solution The easiest way to solve this is to add a <code class="highlighter-rouge">_toJson</code> function to your object, and call that instead of <code class="highlighter-rouge">json_encode</code>: <figure class="highlight"><pre><code class="language-php" data-lang="php"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 5 6 7 8 9 10 11</pre></td><td class="code"><pre><?php public function _toJson() { $properties = $this->getProperties(); $object = new StdClass(); $object->_class = get_class($this); foreach ($properties as $name => $value) { $object->$name = $value; } return json_encode($object); } </pre></td></tr></tbody></table></code></pre></figure> The <code class="highlighter-rouge">getProperties</code> function should return the properties you want to include in the encoding. At it’s simplest it can be <figure class="highlight"><pre><code class="language-php" data-lang="php"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 5</pre></td><td class="code"><pre><?php public function getProperties() { return get_object_vars($this); } </pre></td></tr></tbody></table></code></pre></figure> Then, to call it, just check if the function exists, otherwise just call json_encode: <figure class="highlight"><pre><code class="language-php" data-lang="php"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2</pre></td><td class="code"><pre><?php echo method_exists($object, '_toJson') ? $object->_toJson() : json_encode($object); </pre></td></tr></tbody></table></code></pre></figure> It’s relatively simple to do it as a Decorator as well, which will enable you to give this functionality to multiple classes without the need to duplicate the code. See how I did it in the <a href="https://github.com/backend/Backend-PHP">Backend-PHP</a> framework. Do you have a better way to do it? Say so in the comments. 22seven.com - Folly, or Genius? 2012-01-28T08:37:06+00:00 http://jrgns.net/22seven-com-folly-or-genius/22seven-com-folly-or-genius ##Promising Product Last week Christo Davel, founder of the now defunct <a href="http://mg.co.za/article/2005-11-02-online-bank-20twenty-up-for-sale">twenty20</a> bank, released an online personal accounting service called <a href="http://22seven.com">22seven.com</a>. It <a href="http://www.itweb.co.za/index.php?option=com_content&view=article&id=50971:22seven-goes-live">looked</a> like a great product. Something I was looking for for quite a while: A simple online interface that automatically imports my transaction details, and reports on how much I’m spending on what. I will never use it, though. Why? The app asks for your online banking login details. ##But… In case you don’t know. This is “not a good thing”(tm). As <a href="https://twitter.com/?tw_e=details&tw_i=162563523292041216&tw_p=tweetembed#!/MichaelJordaan/statuses/162563523292041216">tweeted</a> by Micheal Jordaan, CEO of FNB, you’re taking a big risk by doing so. Once you’ve given it to them, it’s out there, somewhere. Yes, they <a href="https://www.22seven.com/security.html">secure</a> the information, but to use it, it needs to become readable at some point. If it can become readable, someone can steal it. From a dishonest employee, to a hacker that found his way on to their servers. The question is, why did they decide to go the route of asking for a user’s banking login details? The banks have been struggling to educate users so that they don’t enter the online banking sites from any other place than the bank’s site, and that they don’t give their login credentials to any third party. As far as I know, the South African banks will refund any money you lost because of online fraud, except if they can prove that you gave your credentials to someone else. And this service is asking for it, wholesale. ##The Problem The problem is that this isn’t the only company that went this route. Sidpayment, a product from Setcom, gives merchants the ability to do online EFTs. The problem is that it’s a Java app that looks over the customer’s shoulder while he’s doing a once off payment through his internet banking. The opportunity for hacks and attacks abound. I’m sure both these knew the risks and were aware of the potential <a href="http://memeburn.com/2012/01/new-startup-22seven-under-fire-from-banks/">backlash</a>, but why did they knowingly proceed with it? There’s a simple answer: They had no choice. The banks in South Africa are walled gardens, with all of your information locked up in it. There’s no way to access it, except through their portals: the branch and online banking. Even FNB, who seems to be embracing newer technologies, doesn’t have anything that even looks like an API. All they have is an accounting app that looks like it was written by (no offence) accountants. So how are third party applications supposed to access a client’s transaction details? Is there even a need for third parties to access people’s transaction information? Of course there is! In a number of other countries you see <a href="http://www.wesabe.com">innovation</a> in <a href="http://mint.com">personal finance</a> because they have access to information. There’s a number of services that track your credit card and spending patterns. These services depend on (sometimes read-only) data they get from banks and financial institutions. ##Come on, Banks! If we are to see innovation and progress in personal accounting in South Africa, the banks need to lead the way by not walling everything off, but by giving accredited and approved third party developers and companies access to API’s. Perhaps 22seven.com’s play was to force the hand of the big banks, and not as stupid as it seems. Setting up Puppet on Ubuntu 10.4 2011-11-16T04:24:06+00:00 http://jrgns.net/setting-up-puppet-on-ubuntu-10.4/setting-up-puppet-on-ubuntu-10.4 I’m trying to use puppet to setup standard dev and (eventually) production environments. As Ubuntu 10.4 is the current LTS release (at least until 12.4), I need to get puppet running on 10.4. The problem is that although the latest stable release for puppet is 2.7.6, puppet in the ubuntu repo is version 0.25.4. Here’s how to firstly set it up, and then upgrade it to the latest version. <h2 id="setup">Setup</h2> First install packages required by Puppet via aptitude <div class="highlighter-rouge"><pre class="highlight"><code>sudo apt-get update sudo apt-get install irb libopenssl-ruby libreadline-ruby rdoc ri ruby ruby-dev </code></pre> </div> And then install puppet and puppet master, also through aptitude <div class="highlighter-rouge"><pre class="highlight"><code>sudo apt-get install puppet puppetmaster </code></pre> </div> You can skip install puppet if your working on the master machine, or skip puppetmaster if your working on a client. I’ll be installing the client and the master on the same machine. You can then follow the following steps from <a href="http://docs.puppetlabs.com/references/0.25.4/configuration.html">the documentation</a> to get the default setup going. Run all commands as root: <div class="highlighter-rouge"><pre class="highlight"><code>#Generate a default config file puppetd --genconfig > etc/puppet/puppet.conf #Generate a default site manifest puppetd --genmanifest > etc/puppet/manifests/site.pp #Add the puppet user and group puppetd --mkusers </code></pre> </div> We also need to tell the client which server it should use as it’s master. Add the following line to <code class="highlighter-rouge">/etc/hosts</code>: 127.0.0.1 puppet Puppet uses the host name <code class="highlighter-rouge">puppet</code> as the default master. You can change this by setting the server config value in <code class="highlighter-rouge">/etc/puppet/puppet.conf</code> You can then test your client setup by running <div class="highlighter-rouge"><pre class="highlight"><code>puppetd --test </code></pre> </div> <h2 id="cert-issues">Cert Issues</h2> At this point I ran into a stupid config issue, where the client complained bombed out with the following error: could not retrieve catalog from remote server: hostname was not match with the server certificate Despite the <a href="http://projects.puppetlabs.com/issues/7224">bad english and uninformative message</a>, I eventually figured out that even though a puppet client by default uses the puppet hostname as its master, the name on the cert that was generated for the hostname isn’t puppet, but the current machine’s name. This mismatch causes the confusion, so either change the server config value, or regen the cert with hostname puppet. I just changed the config value to my machine’s name. This actually negates the need for the added line in <code class="highlighter-rouge">/etc/hosts</code>. PHP Community in South Africa, lack thereof 2011-10-05T05:07:46+00:00 http://jrgns.net/php-community-in-south-africa-lack-thereof/php-community-in-south-africa-lack-thereof I really like working with PHP. It’s a great language despite, a number of shortcomings, and I get to make a living out of writing PHP code. I also love to socialize, and the past few weeks I’ve been looking for a way to combine these two. I’ve looked for conferences or get togethers that I can attend. I looked for online groups or chatrooms about PHP where South Africans gathered. But I couldn’t find any. Well, any active, constructive ones in any case. I see that there’s an active <a href="http://drupal.co.za">Drupal</a> community in and around Jo’burg, so I’ll try to attend some of their meetups. Why Johnny Died (or, How I Finally Realised Why Singletons And Global Variables Are Bad) 2011-09-28T04:06:19+00:00 http://jrgns.net/why-johnny-died-or-how-i-finally-realised-why-singletons-and-global-variables-are-bad/why-johnny-died-or-how-i-finally-realised-why-singletons-and-global-variables-are-bad Johnny (an asthma sufferer and fictitious character) died yesterday. He got an asthma attack, but he could not find his inhaler, and sadly passed away. Usually his mother kept it for him and made sure he used it, but on the first day he ventured out alone, he got an asthma attack, realised that he didn’t have his inhaler with him, and died. What does this have to do with Singletons or Global variables? Read on… I’ve been struggling to get my head around why singletons and global variables are bad. Particularly in the context of unit testing. I’m <a href="http://stackoverflow.com/questions/228590/what-is-the-best-method-for-getting-a-database-connection-object-into-a-function/228715#228715">quite a fan</a> of a global static class that can manage certain resources for you. But if you just look around a bit, you’ll find a <a href="http://www.webapper.com/blog/index.php/2008/01/23/evils-of-global-variables-when-unit-testing/">few</a> <a href="http://sebastian-bergmann.de/archives/882-Testing-Code-That-Uses-Singletons.html">resources</a> around the fact that they are a bad idea. I read up quite a bit on why exactly globals and singletons are bad, but couldn’t find any convincing explanations. Until yesterday, when I read this <a href="http://stackoverflow.com/questions/1020312/are-singletons-really-that-bad">question</a> and <a href="http://stackoverflow.com/questions/1020312/are-singletons-really-that-bad/1020384#1020384">answer</a> on stackoverflow. Particularly <blockquote> every class can potentially depend on a singleton, which means the class can not be reused in other projects unless we also reuse all our singletons </blockquote> In other words, if you want your classes / objects to be modular and easily transportable and reusable in other code bases, you cannot use global variables or singletons. To properly unit test, you need to use your class in a clean environment, with only it’s stated dependencies injected into it on creation. Otherwise there’s no guarantee that your tests will fail or succeed consistently. Doing this also promotes code reuse, as you know that the global state won’t be affected in some averse way. What does this have to do with Johnny? Look at him as an object of the class Human, who has a dependency on an Inhaler object. His mother is a global variable or class that supplies him with the Inhaler when he needs it. All fine and dandy if Johnny operates within the context of his mother’s home. Take him out of that context, and he dies when he needs the inhaler. Rather give him the inhaler, and let him keep it in his pocket. Dependency injected! A PHP Virtual Host 2011-09-13T03:18:03+00:00 http://jrgns.net/php-virtualhost/php-virtualhost I hate repetitive work. I love solving a puzzle. So it often happens that I’d spend 2 hours writing a script to solve a repetitive problem, than just taking the 2 minutes to do it manually. Creating apache config files are tedious and boring, so I wrote a PHP script that emulates Virtual Hosting. You can get the code on <a href="https://github.com/jrgns/php-virtualhost">GitHub</a>. A Warning: The code is not production ready. Use it at your own risk. Here’s what it does: An <code class="highlighter-rouge">.htaccess</code> file <a href="?q=content/redirect_request_to_index">redirects all requests</a> to the <code class="highlighter-rouge">index.php</code> file, which sits in <code class="highlighter-rouge">/var/www</code>. We get the host from the <code class="highlighter-rouge">$_SERVER</code> variable, and check if a folder for that host exists. <figure class="highlight"><pre><code class="language-php" data-lang="php"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 5 6 7 8</pre></td><td class="code"><pre><?php $host = $_SERVER['HTTP_HOST']; $folder = getcwd() . '/' . $host . '/'; if (!file_exists($folder)) { no_host(); die; } ?> </pre></td></tr></tbody></table></code></pre></figure> So if we wanted to server jrgns.net from it, the folder <code class="highlighter-rouge">/var/www/jrgns.net</code> should exist on the server. If it doesn’t, a generic message is displayed. Courtesy of the <code class="highlighter-rouge">.htaccess</code> file, the whole URL, except the host name, gets passed to the script in the <code class="highlighter-rouge">f</code> query variable. As we might be requesting a stylesheet such as jrgns.net/styles/basic.css, the virtual host should pick that up. If no file is requested, check for and serve an index file. <figure class="highlight"><pre><code class="language-php" data-lang="php"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29</pre></td><td class="code"><pre><?php if (empty($_REQUEST['f'])) { $indexes = array( 'index.php', 'index.html', 'index.htm', 'main.php' ); foreach($indexes as $index) { if (file_exists($folder . $index)) { chdir($folder); include($folder . $index); die; } } } else { $file = $_REQUEST['f']; if (file_exists($folder . $file)) { $info = explode('.', $file); switch(end($info)) { case 'js': header('Content-Type: text/javascript'); break; case 'css': header('Content-Type: text/css'); break; } readfile($folder . $file); die; } } ?> </pre></td></tr></tbody></table></code></pre></figure> I use a very rudimentary way to get the file extension, and then send the content header. There are probably better ways to do this, but it works. The rest is just the generic message that gets displayed if we don’t know what you’re looking for. <figure class="highlight"><pre><code class="language-php" data-lang="php"><table style="border-spacing: 0"><tbody><tr><td class="gutter gl" style="text-align: right"><pre class="lineno">1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19</pre></td><td class="code"><pre> <?php function no_host() { global $host; ?><!DOCTYPE html> <html> <head> <title>Unknown Domain: <?php echo $host ?></title> </head> <body> <div> <h1>Unknown Domain: <?php echo $host ?></h1> </div> </body> </html> <?php } no_host(); ?> </pre></td></tr></tbody></table></code></pre></figure> Like I said. Simple! A JavaScript trim function 2011-07-18T04:55:32+00:00 http://jrgns.net/js-trim/js-trim A simple enough <code class="highlighter-rouge">trim</code> function for JavaScript: <div class="highlighter-rouge"><pre class="highlight"><code>function trim(string) { return string.replace(/^\s*|\s*$/g, '') } </code></pre> </div>  The regular expression translates as <ul> <li><code class="highlighter-rouge">^</code> - At the beginning of the line</li> <li><code class="highlighter-rouge">\s*</code> - Take all the white space you can find, if there is any</li> <li><code class="highlighter-rouge">|</code> - OR</li> <li><code class="highlighter-rouge">\s*</code> - Take all the white space you can find, if there is any</li> <li><code class="highlighter-rouge">$</code> - At the end of the line</li> </ul> The <code class="highlighter-rouge">g</code> parameter at the end ensures that all instances are replaced. The replace function uses the regular expressions to replace the white space it found at the beginning or the end of the string with an empty string. Easy enough! Quick Guide To Creating A Website Quickly 2011-07-11T21:31:53+00:00 http://jrgns.net/quick_guide_to_creating_a_website_quickly/quick_guide_to_creating_a_website_quickly The last two weeks I had the privilege of staying in a house on the beach near Mosselbay. This time of the year whales start coming into the bay, and, if you’re lucky, you can spot them breaching or displaying their flukes or fins. This inspired me to create <a href="http://whalespotter.co.za">WhaleSpotter</a>, a site that uses Twitter to help people spot whales along South Africa’s south coast. It took me less than 2 hours to do it. Here’s how. ###Know exactly what you want to Do This speaks for it self. Spend a few minutes to lay out what you want, and how you want to achieve it. If you need to write it down, explain it to someone or whiteboard it, do it. This will save a lot of time later in the process, as you won’t track back or waste time with features or ideas that doesn’t fit. ###Keep it Simple Keep the number of features down to a minimum. If you can fit everything on one page, do. This is quick and dirty, not long and elaborate. You can expand it later. Your aim now is to get something up and running. ###Know and use your Tools Frameworks cut down a lot in development time. They provide functionality and structure so that you don’t have to work on it. I dev’ed my own MVC framework, <a href="http://backend-php.net">Backend</a>, so I used that. It provides a nice templating system, along with basic configuration, caching and utility functions. Use what ever you’re comfortable with, as long as it reduces the work you have to do, not increase it. ###Use Resources wisely Other resources can also cut down on time taken to get your site up. I used <a href="http://images.google.com">Google Images</a> to get a logo. Look at <a href="http://themeforest.net/">Themeforest</a> for a design that will work for your site. Copy and paste code from your own projects or open source projects on the web. The site was in part inspired by <a href="http://replaceawordinafamousquotewithduck.com/">http://replaceawordinafamousquotewithduck.com/</a>, and I copied some of the twitter parsing code from the <a href="https://github.com/duckduckgo/replaceawordinafamousquotewithduck">GitHub repo</a>. ###So Know where you’re going. Keep it simple. Know your tools and options. Outsource as much as possible. Good Developers Think like a Startup 2011-05-05T08:25:54+00:00 http://jrgns.net/good_developers_think_like_a_startup/good_developers_think_like_a_startup I have the privilege to be the Head of Development of <a href="http://www.brandedinternet.co.za">Branded Internet</a>, a white label ISP in South Africa. I also like to read and think about how startups work, and what they do to make them successful. As I see it, the basic recipe for a startup is: <ol> <li>Find a niche market or problem</li> <li>Find a revolutionary / novel way to solve that problem</li> <li>Build the solution</li> <li>Market it</li> <li>See the money roll in.</li> </ol> If you see the company you’re developing for as your client or target market, software development follows basically the same recipe. <h3 id="find-a-problem">Find a problem</h3> Usually people come to the developers of a company with a problem. I need App X to do Y and Z. App C doesn’t do D correctly. I’m not very good at spotting problems that are potentially money making, so I’m quite happy to be a developer where people come to me looking for solutions. That’s the first step sorted. <h3 id="find-a-way-to-solve-the-problem">Find a way to solve the problem</h3> <img src="http://lorempixum.com/200/300/abstract/" style="float: right;" /> Startups depend on the fact that they solve previously unsolved problems, or they solve problems in such a way that people want to throw money their way for the privilege to use the startup’s solution. I suspect that this may just be one of the reasons why developers at bigger companies don’t necessarily produce quality code: There’s no incentive to do it quickly, elegantly or intelligently. Just get it done. If it takes you three times longer than projected with a code base larger than the Kalahari, who cares. It’s done, and you got your salary at the end of the month. If a startup thinks like that, it’s dead in the water. Solving problems better is always better. Yes, there’s major time and money constraints on most developers, and elegance isn’t always achievable, but good developers should always be open to solve problems in a novel way. Don’t be a mill churning out code, be a creator of great code! <h3 id="build-the-solution">Build the Solution</h3> Do what you do. Be a thinker, then a typist. And do it well. <h3 id="market-it">Market it</h3> Having built the solution is not where it stops. For most developers, it’s where they wished it stopped, because, let’s face it, if we liked speaking to people, we wouldn’t have become developers. But unfortunately you as a developer need to take it further. You need to market what you’ve done. A product that no one knows or knows how to use doesn’t exist. You need to tell your superior and your peers about the fact that the project is up and running. You need to document the code and the use cases so that other developers (including your future self) knows what’s going on. You need to train your clients (the users) who originally requested your help on using the project. You need to get feedback from your clients on if this solves their problem, and go back to the code if it doesn’t. <h3 id="reap-the-rewards">Reap the Rewards</h3> I find it immensely rewarding to be able to solve problems for people. Yes, the salary is great, but there’s times that I will literally code for free. Not just because I like solving problems, but because I know that I can make someone’s work day less frustrating. You get the gold and the golden glow. <h3 id="tldr">tl;dr</h3> By thinking or your rote work in terms of a startup, where you need to be agile and more than just a normal developer, going out of your comfort zone and speaking to people, your average work can become great work, and your worth will be measured in more than just a salary check. Why Object Orientation is Great 2011-02-08T07:00:21+00:00 http://jrgns.net/why_oo_is_great/why_oo_is_great I’m working on some statistics surrounding <a href="http://www.brandedinternet.co.za">BI’s</a> API Documentation coverage: Just how much user documentation have we written? When I started out, the number was abysmally low: 11% of the 1300 functions. Not good. But I didn’t feel too bad when I realized that because it’s written as an Object Orientated system, I can very quickly improve the coverage. As I went through the undocumented functions, it became clear that a lot of them are defined in some of our base classes. So, by documenting them in the base classes, I will immediately improve the documentation coverage for a whole bunch of classes. I tried it with one function, and lo and behold, the percentage jumped with 6 points! So, what do we learn from this? Object orientation is powerful when you implement changes deep down in the object stack, as it propagates through the whole stack. It also reduces the work load (most of the time) as there’s no need to repeat yourself. The downside to consider is that the documentation written is very general (as it should be in a base class), and will probably be overwritten in some of the sub classes. But that’s Object Orientation: Solve 80% of the problem with 20% of the code, and the other 20% of the problem with the other 80% of the code! Getting Started with node.js 2010-10-11T06:31:40+00:00 http://jrgns.net/getting_started_with_node_js/getting_started_with_node_js Looking for a new challenge, I started a small project in node.js. Here’s a few simple tips on how to get going quickly. <h2 id="its-javascipt">It’s JavaScipt</h2> Yeah, that’s obvious, but sometimes stating the obvious is necessary. Node.js is essentially server side JavaScript. This means that all your JS knowledge and tutorials and manuals will be usefull once you get stuck. <h2 id="its-event-driven">It’s Event Driven</h2> Node.js is an event driven framework, which can take some getting used to, but it is very powerful in the long run. You can write straight forward bottom to top code, but then you’ll be missing out on the real power of node.js. An example: <div class="highlighter-rouge"><pre class="highlight"><code>//Get the events module var events = require('events') , fs = require('fs'); //Create a new event object var file_emitter = new events.EventEmitter(); //Create a listener for our event object var newFileListener = file_emitter.on('found', function(file) { //Do something to the found file } //Check for files in a folder var files = fs.readdirSync('/some/folder'); files.forEach(function(file) { //Emit (Trigger) the "found" event on our event object file_emitter.emit('found', dest); }); </code></pre> </div> <h2 id="multiple-files">Multiple Files</h2> Any good coder knows that putting your code in multiple files is a quick win for organized coded. Here’s how to do it in node.js: <div class="highlighter-rouge"><pre class="highlight"><code>//In utils.js exports.trim = trim; function trim(string) { return string.replace(/^\s*|\s*$/g, '') } //In the file you want to use the trim function var utils = require('./utils') </code></pre> </div> The <code class="highlighter-rouge">.js</code> part of the file is automatically added/detected. This is to include a file in the same folder as the calling file, but you can easily just add the correct path traversals to get to other files. The important part in the file being included is setting the function name in the <code class="highlighter-rouge">exports</code> Object. <h2 id="file-io">File I/O</h2> Files are all treated as streams in node.js. This means that you can start acting on the data in a file before node has even finished reading the file. <div class="highlighter-rouge"><pre class="highlight"><code>//Open a file as a Readable stream stream = fs.ReadStream(file); stream.setEncoding('ascii'); stream.on('data', function(data) { //Do something to the file's data mid stream }); stream.on('close', function () { //Do something to the file once it's finished reading } </code></pre> </div> Writing to a file is just as simple <div class="highlighter-rouge"><pre class="highlight"><code>stream = fs.createWriteStream(filename, { 'encoding': 'base64' }); stream.write(data); stream.end(); </code></pre> </div> <h2 id="want-more">Want More?</h2> See <a href="http://github.com/jrgns/parser_email">parser_email on github</a> for the full source of the examples given. Some useful sites: <ul> <li> <table> <tbody> <tr> <td><a href="http://nodejs.org/">Official Site</a></td> <td><a href="http://nodejs.org/api.html">Documentation</a></td> <td><a href="http://github.com/ry/node">Source</a></td> </tr> </tbody> </table> </li> <li><a href="http://debuggable.com/posts/understanding-node-js:4bd98440-45e4-4a9a-8ef7-0f7ecbdd56cb">Understanding node.js</a></li> <li> <a href="http://howtonode.org/">How to Node</a> </li> </ul> Email Parser for node.js 2010-10-07T09:53:55+00:00 http://jrgns.net/node_parser_email/node_parser_email I need to parse Emails in node.js and after looking around on the web, I didn’t find much. Oh, no, I’ll have to write it myself… :) The source is hosted on <a href="http://github.com/jrgns/parser_email">GitHub</a> . Here’s an example: <div class="highlighter-rouge"><pre class="highlight"><code>var fs = require('fs') , sys = require('sys') , em_parse = require('./parser_email') stream = fs.ReadStream(file); stream.setEncoding('ascii'); stream.on('data', function(data) { mail += data; }); stream.on('close', function () { parser = em_parse.parser_email(); parser.setContent(mail); parser.parseMail(); }); </code></pre> </div> Why I Code 2010-04-15T08:56:23+00:00 http://jrgns.net/why_i_code/why_i_code A a student I went to a career psychologist to explore my career needs and drives, and got told that I like solving problems in a creative way. As the son of an engineer and a very artsy mother, that kind of made sense. Seeing as I was already into computers and programming, I chose coding as my problem solving mechanism. Through the years I learned that I loved, never mind liked, solving problems this way. I wrote a <a href="http://backend-php.net">framework</a>, because I saw a lot of unstructured and repetitive code in my projects. I dev’ed <a href="http://whatsupinmelville.co.za">What’s up in Melville</a> because I needed information about the area I lived in. I saw <a href="http://news.ycombinator.com">Hacker News</a>, and wanted the same site, but for South Africa, so I dev’ed <a href="http://zacoders.net">ZA Coders</a>. All (hopefully) creative solutions to problems I faced. I am now Head of Development at <a href="http://www.brandedinternet.co.za">Branded Internet</a>, and I’m still very passionate about solving problems through code. Any dev department at a company whose main aim is not to produce software, is to help the rest of the company to solve their problems through code. Finances needs to regularly generate a certain report, Dev scripts the code, and gives them the link. Support needs to information on an account, Dev scripts the code, and gives them the link. Sometimes Dev departments will write code to help themselves, but most of the work will be done for other departments. So the Dev department needs to communicate, needs to interact, so that they know which problems to solve to help the other departments do their job. Development departments exists to help other departments. In other words, it’s not about me. Twitter API Request Entity too large / HTTP 413 error 2010-03-30T07:12:09+00:00 http://jrgns.net/twitter_api_request_entity_too_large_http_413_error/twitter_api_request_entity_too_large_http_413_error I’ve setup <a href="http://zacoders.net">zacoders.net</a> that it will automatically tweet new RFC’s. A while back this functionality stopped working. Today I investigated a bit further, and here’s what I found. When posting an update, the Twitter API responded with a 413 HTTP error code, and the following text: <div class="highlighter-rouge"><pre class="highlight"><code>Request Entity Too Large The requested resource /1/statuses/update.json does not allow request data with POST requests, or the amount of data provided in the request exceeds the capacity limit. </code></pre> </div> I realized that the <a href="http://backend-php.net">backend</a>’s OAuth utility sent through POST requests with the parameters included in the URI, just like a GET request: POST HTTP/1.1 http://api.twitter.com/1/statuses/update.json?parameters=here What baffled me is that I only got this error when running it on my live server, not on my testing local server. I suspect it might be a difference in the Curl libraries, but I’m not sure. To fix it, I just updated the code to only add the parameters to the URI when it was a GET, and to POST the data when it was a POST, and voila, it’s working again. How to remove the www from your URL 2010-03-26T08:03:29+00:00 http://jrgns.net/remove_www_from_url/remove_www_from_url If you like simplicity, you might like to simplify your website’s URL. You can do this by adding the following in your <code class="highlighter-rouge">.htaccess</code> file, or in the <code class="highlighter-rouge">VirtualHost</code> declaration on your Apache setup. <div class="highlighter-rouge"><pre class="highlight"><code>RewriteEngine on RewriteCond %{HTTP_HOST} ^www.mysite.co.za$ [NC] RewriteRule ^(.*)$ http://mysite.co.za/$1 [L,R=301] </code></pre> </div> The rewrite module for apache needs to be installed for this to work. Remember that you can tell Google to list all links to your site with / without the www by setting that in the <a href="http://www.google.com/webmasters/tools/">Webmaster Tools</a>. New look! 2010-03-24T23:05:40+00:00 http://jrgns.net/new_look/new_look At long last I’ve been able to find the time to port my website from drupal to <a href="http://backend-php.net">backend</a>. Backend is a PHP framework I wrote myself and open sourced to get it out there. I’ve also redesigned the site, made it simpler and hopefully more usable. I’ll be adding more information as I go along. SQLite Viewer 2010-03-18T16:28:30+00:00 http://jrgns.net/sqllite_view/sqllite_view I’m playing around with <a href="http://developers.google.com/gears/">Google Gears</a>, doing data imports, and generally playing around. I’m quite visiually oriented, so I need to SEE that the data has been added to a table. For my conventional PHP / MySQL apps, I usually use phpMyAdmin or the MySQL query browser, but for Gears developmen, which uses SQLite, I needed to use something else. I wrote a small JavaScript / Gears app which enables you to view the contents of a SQLite database. All you need to use it, is to download the file, and unzip it into the folder where your web app sits. And no, you won’t be able to use it for a remote DB. You need at least FTP access to the website you want to look at, and it will only show you your local data. You will also need to know the name of the database. At the moment I can see no simple way to supply the user with a listing of DB’s, so the app prompts the user for the DB name. The app currently links to the scripts and styles on my host, which is fine for light use. If you plan to use this app extensively, please start hosting the files on your own host and update the HTML! Update On Backend 2010-03-18T16:23:37+00:00 http://jrgns.net/Update on Backend/Update on Backend Bakend: PHP framework, as it’s now called, now has it’s own website! Check it out at <a href="http://backend-php.net">http://backend-php.net</a>. Follow backend on Twitter at <a href="http://twitter.com/backend_php">@backend_php</a>! Backend 100 Revisions 2010-03-18T16:20:29+00:00 http://jrgns.net/backend_100_revisions/backend_100_revisions I’ve been working on <a href="http://backend-php.net">backend</a>, a PHP framework the last couple of months, and it’s really starting to take shape. I’ve learned a lot about what one can do by coding smarter and the advantages open source code gives on. After getting somewhat disheartened a few weeks back, I decided to take a look at how <a href="http://codeigniter.com/">CodeIgnitir</a> does things, and wow, it blew my mind. After spending some time with their code base, I had so many ideas I wanted to implement in backend and my thoughts around it was instantly rejuvinated. Problems that stopped me from progressing had new solutions, and the momentum I gathered from that helped to solve some problems that didn’t have any solutions directly at hand. So, backend just hit it’s 100th revision, somewhat arbitrarily, as I reset the branch at some point, and don’t always follow the exact same commit methodology, but hey, it looks like a milestone, I’ll treat it as one. What has changed? URL mapping has improved a lot (thanks to CodeIgnitir). <code class="highlighter-rouge">?q=controller/action/param1/param2</code> now maps to <code class="highlighter-rouge">Controller::action(param1, param2)</code> without much effort. This has also helped to work towards being a RESTful API. The installation process is now very smooth and stable. The setting up of data sources need some work, but once that is done, getting the application up and running and fine tuning it is a breeze. The beginning of a Links structure with named link lists can be managed. The primary use will be for primary and secondary links for a website. I’ve managed to implement tags for content, as well as RSS feeds for content as a whole, or content in a tag. The ease with which I could do it really gives me hope that backend will make my coding life easier! All around polish and stability. Especially around the HTML templating and rendering engine. Where to from here? The permissions / security infrastructure still needs some work. I’ve managed to automatically install most of the default permissions, but there’s no useful management interface yet. I want to start using backend on my personal site (this one) and on another site I’ve got in the works, called <a href="http://zacoders.net">zacoders.net</a>. More on that later. My take on single or multiple returns 2010-03-18T16:17:11+00:00 http://jrgns.net/my_take_on_single_or_multiple_returns/my_take_on_single_or_multiple_returns Originally posted on <a href="http://stackoverflow.com/questions/36707/should-a-function-have-only-one-return-statement/1276951#1276951">StackOverflow</a> I force myself to use only one return statement, as it will in a sense generate code smell. Let me explain: <div class="highlighter-rouge"><pre class="highlight"><code><?php function isCorrect($param1, $param2, $param3) { $toret = false; if ($param1 != $param2) { if ($param1 == ($param3 * 2)) { if ($param2 == ($param3 / 3)) { $toret = true; } else { $error = 'Error 3'; } } else { $error = 'Error 2'; } } else { $error = 'Error 1'; } return $toret; } ?> </code></pre> </div> (The conditions are arbritary…) The more conditions, the larger the function gets, the more difficult it is to read. So if you’re attuned to the code smell, you’ll realise it, and want to refactor the code. Two possible solutions are: <ul> <li>Multiple returns</li> <li>Refactoring into separate functions</li> </ul> Multiple Returns <div class="highlighter-rouge"><pre class="highlight"><code><?php function isCorrect($param1, $param2, $param3) { if ($param1 == $param2) { $error = 'Error 1'; return false; } if ($param1 != ($param3 * 2)) { $error = 'Error 2'; return false; } if ($param2 != ($param3 / 3)) { $error = 'Error 3'; return false; } return true; } ?> </code></pre> </div> Separate Functions <div class="highlighter-rouge"><pre class="highlight"><code><?php function isEqual($param1, $param2) { return $param1 == $param2; } function isDouble($param1, $param2) { return $param1 == ($param2 * 2); } function isThird($param1, $param2) { return $param1 == ($param2 / 3); } function isCorrect($param1, $param2, $param3) { $toret = false; if (!isEqual($param1, $param2) && isDouble($param1, $param3) && isThird($param2, $param3) ) { $toret = true; } return $toret; } ?> </code></pre> </div> Granted, it is longer and a bit messy, but in the process of refactoring the function this way, we’ve <ul> <li>created a number of reusable functions,</li> <li>made the function more human readable, and</li> <li> the focus of the functions is on why the values are correct. </li> </ul> Firefox requests a page twice? 2010-03-18T16:13:43+00:00 http://jrgns.net/firefox_requests_a_page_twice/firefox_requests_a_page_twice I’m busy creating a site for a client who wants to know when certain internal links get clicked. Not a problem, I just insert a record into a table every time the links they want to know about get accesesd. Only problem is, when I check the table, I notice that every page impression gets logged twice. Now, this happened before, where I also needed to implement some page tracking, and it ended up being an image or something with src=””, and Firefox interpreting it as the same location as the current page (as with form with action=””). Don’t know if that is according to standards, either way, it messes up my page tracking. The simple solution is to find any images or HTML elements with source attributes that are empty. Only problem is I couldn’t find any this time. So, by taking out chunks of code, I eventually tracked it to srcover attributes in some of the images. Just as a side note, I don’t use srcover, the designers use them, and unfortunately I must make do with what the designers give me. So why does Firefox (I didn’t test on other browser) do this on srcover? Firebug shows that the images specified in srcover are downloaded, and the mouseovers work, so it’s not that the images are missing or anything. I’ll investigate when I have some time, and report back… Update: I found this page at <a href="http://www.experts-exchange.com/Web_Development/Web_Languages-Standards/PHP/PHP_Databases/Q_23101341.html">Experts Exchange</a>, but hey, they want me to pay to see it. No chance of that happening. If anyone is a member, and you don’t want money to share some wisdom, please do! Update number two: I fount <a href="http://forums.mozillazine.org/viewtopic.php?p=3022038">this page</a> which only talks about the empty src tag, but it also quotes the HTML spec that says that empty src tags refer to the same page their in. So why does srcover still re-request my page? Update number final: Ok, so it wasn’t Firefox’s fault, but the javascript that did the rollover images. When it preloades, it doesn’t check for empty src attributes, it just loads them. As it checks for both srcover and srcdown, which aren’t necessarily specified, it gets quite a lot of empty tags. I guess that Firefox uses a cached page for the rest of the requests. The javascript library is called imagerollover.js by Adam Smith. I couldn’t find a working website, so I don’t know if I have the latest version, but I’ll upload my modified version… Parse Http Accept Header 2010-03-18T16:10:35+00:00 http://jrgns.net/parse_http_accept_header/parse_http_accept_header I want to generate CSS on the fly for a personal project. I’m doing this by redirect all requests for files that does not exist to index.php, and then, if the request is for a stylesheet, index.php generates it. To identify a request for a stylesheet, we need to parse the Accept header in the HTTP Request. I’ve grouped all the functions together in a Parser class. This function retrieves the header, and then determines the mime type, precedence, and tokens for each type given. It’s then sorted according to <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html">RFC 2616</a> specifications. <div class="highlighter-rouge"><pre class="highlight"><code><?php public static function accept_header($header = false) { $toret = null; $header = $header ? $header : (array_key_exists('HTTP_ACCEPT', $_SERVER) ? $_SERVER['HTTP_ACCEPT']: false); if ($header) { $types = explode(',', $header); $types = array_map('trim', $types); foreach ($types as $one_type) { $one_type = explode(';', $one_type); $type = array_shift($one_type); if ($type) { list($precedence, $tokens) = self::accept_header_options($one_type); list($main_type, $sub_type) = array_map('trim', explode('/', $type)); $toret[] = array('main_type' => $main_type, 'sub_type' => $sub_type, 'precedence' => (float)$precedence, 'tokens' => $tokens); } } usort($toret, array('Parser', 'compare_media_ranges')); } return $toret; } ?> </code></pre> </div> This helper function parses the options for each mime type, determining the precedence and tokens. It returns the precidence as a float, and an array of tokens. <div class="highlighter-rouge"><pre class="highlight"><code><?php public static function accept_header_options($type_options) { $precedence = 1; $tokens = array(); if (is_string($type_options)) { $type_options = explode(';', $type_options); } $type_options = array_map('trim', $type_options); foreach ($type_options as $option) { $option = explode('=', $option); $option = array_map('trim', $option); if ($option[0] == 'q') { $precedence = $option[1]; } else { $tokens[$option[0]] = $option[1]; } } $tokens = count ($tokens) ? $tokens : false; return array($precedence, $tokens); } ?> </code></pre> </div> This function is used to sort the array of mime types we parsed from the header. At the moment it doesn’t check any tokens, but this code can easily be added on line 6. <?php private static function compare_media_ranges($one, $two) { if ($one['main_type'] != '*' && $two['main_type'] != '*') { if ($one['sub_type'] != '*' && $two['sub_type'] != '*') { if ($one['precedence'] == $two['precedence']) { if (count ($one['tokens']) == count ($two['tokens'])) { return 0; } else if (count ($one['tokens']) < count ($two['tokens'])) { return 1; } else { return -1; } } else if ($one['precedence'] < $two['precedence']) { return 1; } else { return -1; } } else if ($one['sub_type'] == '*') { return 1; } else { return -1; } } else if ($one['main_type'] == '*') { return 1; } else { return -1; } } ?> Backend-PHP Framework 2010-03-18T08:46:32+00:00 http://jrgns.net/backend_intro/backend_intro <div class="notice"> <h3>Update: 21/11/2011</h3> Backend has undergone a major rewrite. Get the latest info <a href="/content/backend-core-a-restful-mvc-php-framework">here</a>, and get the code on <a href="https://github.com/jrgns/backend-core">GitHub</a> </div> In the past few years I’ve been working on a PHP framework / backend. Sometimes out of frustration with copying and pasting code, but usually because I’m to lazy to write a lot of code, so decide to write a little bit of code to do a lot… And then I write some more code, to do a little bit more, and before you know it, it’s yet another PHP framework. I tried to get some public input on the <a href="http://launchpad.net/yaphpfter">project</a> once before, but that went nowhere, and I left it there - until of course I had to do some work that required the same type of thing, and now I’m at it again. What makes backend different from my previous <a href="http://www.codinghorror.com/blog/archives/000576.html">failed</a> experiment? Well, the fact that this is now the 7th or 8th time that I’ve rehashed the concept, and that this time, I’ve actually put some thought and effort into it. No, it’s not perfect, and it won’t solve the world’s problems, but it better and simpler, and hopefully more useful. One concept that I definitely want to run with in backend, is the concept of backend as a web service, as opposed to a framework or a content / data management system. Backend provides a client, as in a browser or an application, with the requested data, in a format that the client can understand. Be it a human readable web page, JSON, XML or even PHP variables. This struck me as important after reading <a href="http://tomayko.com/writings/rest-to-my-wife">this</a> by Ryan Tomayko: <blockquote> Wife: A web page is a resource? Ryan: Kind of. A web page is a “representation” of a resource. Resources are just concepts. URLs—those things that you type into the browser… Wife: I know what a URL is.. Ryan: Oh, right. Those tell the browser that there’s a concept somewhere. A browser can then go ask for a specific representation of the concept. Specifically, the browser asks for the web page representation of the concept. Wife: What other kinds of representations are there? Ryan: Actually, representations is one of these things that doesn’t get used a lot. In most cases, a resource has only a single representation. But we’re hoping that representations will be used more in the future because there’s a bunch of new formats popping up all over the place. Wife: Like what? Ryan: Hmm. Well, there’s this concept that people are calling “Web Services”. It means a lot of different things to a lot of different people but the basic concept is that machines could use the web just like people do. </blockquote> So it’s important for me to make it easy for web developers to easily provide data in different formats. Backend can already provide any client with data in 4, yes 4, different formats: <ul> <li>HTML, or a human readable webpage</li> <li><a href="http://json.org/">JSON</a>, the new darling of AJAX fans</li> <li><a href="http://www.php.net/serialize">(PHP) Serialized byte stream representations</a></li> <li><a href="http://www.php.net/var-export">PHP variables</a>, as created by var_export</li> </ul> This will hopefully increase backend’s usefulness as a web service. And yes, I know, XML is missing, but of those 4, HTML was necessary, although not necessarily easy, and the other three were extremely easy to implement, as they are all products of simple PHP functions. One day when someone uses backend and actually needs XML, we can implement it. (Notice that I said we, not I - I rarely use XML, so I won’t know the requirements, so if I do it alone, it will probably be useless). At the moment backend is on version 0.1.2.1 in my personal repo. All that means is that I’ve decided I’ve reached a point where I can actually release the code to the public, and made some improvements, fixes and changes. When will it be marked as version 1.0? When we have the following: <ol> <li>A fully fledged and integrated admin space 2, Easy and intuitive setup / install system</li> <li>Easy and intuitive update system</li> <li>An intuitive plugin system</li> <li>Intuitive and function DB Objects<</li> <li>The ability to easily implement different data formats</li> <li>The ability to use multiple and different data sources (MySQL, SQLite, flat files, etc.)</li> <li>Working and secure user access control</li> <li>Either a REST-full implementation, or a roadmap towards it</li> <li>Something that makes coding easier</li> </ol> That’s ten requirements for me to release backend v1.0… What’s already done? <ul> <li>Semi working DB Objects</li> <li>Output of different data formats using views</li> <li>Simple templating / caching system</li> <li>Simple user access control</li> <li>CRUD, actually, just CRU actions. I haven’t gotten round to the Deletes yet.</li> </ul> I’m using <a href="http://jquery.com/">jQuery</a> as a JavaScript backend, and <a href="http://www.blueprintcss.org/">blueprint</a> for my CSS needs. For now it’s under the <a href="http://www.opensource.org/licenses/eclipse-1.0.php">EPL</a>, but I may switch to the MIT license at some point. I’m using MySQL exclusively at this point, but am hoping to add support for other DB’s / sources eventually. I’ll be putting the code on <a href="http://launchpad.net/backend">Launchpad</a> soon, so if you want to contribute, or comment, please do! Importing a file into a Google Gears DB using PHP 2010-03-18T08:45:19+00:00 http://jrgns.net/import_to_google_gears/import_to_google_gears I’m playing around with <a href="http://developers.google.com/gears/">Google Gears</a>, particularly the database part, just to see what it’s usefull for, and if I can use it in an app I’m working on. At some point I realised that I will need to import data at some point. Here’s how I did it:  By using an HTML uploading form, and PHP to process the upload, you can create a JSON object which can be used to import data into whatever Gears DB you have. Use whatever you use to put all of the data into one array. The source can be a DB or a file upload, come to think of it. Convert the data to JSON using json_encode, and assign it to a JavaScript variable: <code>var data = <?php echo json_encode($data) ?>;</code> After that, check that the table exists, loop through the data, and insert! The complete code below: The test data to import: <pre> "one two",2,1 "three four",3.4,2 </pre> And the test page, HTML and PHP together: <div class="highlighter-rouge"><pre class="highlight"><code><?php $data = false; if ($_SERVER['REQUEST_METHOD'] == 'POST') { $data = array(); if (!empty($_FILES['to_import']['tmp_name']) && is_readable($_FILES['to_import']['tmp_name'])) { $fp = fopen($_FILES['to_import']['tmp_name'], 'r'); while (($row = fgetcsv($fp)) !== false) { $data[] = $row; } fclose($fp); } $data; } ?> <html> <head> <title>JSON / Gears import Test</title> </head> <body onload="do_import()"> <form method="post" action="" enctype="multipart/form-data"> <input type="file" name="to_import"> <input type="submit"> </form> <script src='gears_init.js'></script> <script> var data = <?php echo json_encode() ?>; function do_import() { var db = google.gears.factory.create('beta.database'); db.open('database-test'); db.execute('CREATE TABLE IF NOT EXISTS Import' + ' (Phrase text, Amount float, Something int)'); if (data) { for (var i = 0; i < data.length; i++) { db.execute('INSERT INTO Import Values(?, ?, ?)', data[i]); } rs = db.execute('SELECT * FROM Import'); while(rs.isValidRow()) { alert(rs.field(0) + ' - ' + rs.field(1) + ' - ' + rs.field(2)); rs.next(); } rs.close(); } } </script> </body> </html> </code></pre> </div> Redirect All Requests To Index.php Using .htaccess 2010-03-18T08:42:15+00:00 http://jrgns.net/redirect_request_to_index/redirect_request_to_index In one of my pet projects, I redirect all requests to index.php, which then decides what to do with it: ###Simple Example This snippet in your .htaccess will ensure that all requests for files and folders that does not exists will be redirected to index.php: <div class="highlighter-rouge"><pre class="highlight"><code>RewriteEngine on RewriteCond %{REQUEST_FILENAME} !-d RewriteCond %{REQUEST_FILENAME} !-f RewriteRule . index.php [L] </code></pre> </div> This enables the rewrite engine: <div class="highlighter-rouge"><pre class="highlight"><code>RewriteEngine on </code></pre> </div> This checks for existing folders (-d) and files (-f): <div class="highlighter-rouge"><pre class="highlight"><code>RewriteCond %{REQUEST_FILENAME} !-d RewriteCond %{REQUEST_FILENAME} !-f </code></pre> </div> And this does the actual redirecting: <div class="highlighter-rouge"><pre class="highlight"><code>RewriteRule . index.php [L] </code></pre> </div> ###Extended Example You can extend this to pass the requested path to the index.php file by modifying the <code class="highlighter-rouge">RewriteRule</code> to the following: <div class="highlighter-rouge"><pre class="highlight"><code>RewriteRule ^(.*)$ index.php?q=$1 [L,QSA] </code></pre> </div> <ul> <li>The <code class="highlighter-rouge">^(.*)$</code> part tells the rewrite module that we want to pass down the whole requested path as one parameter.</li> <li>The <code class="highlighter-rouge">QSA</code> part tells the module to append any query strings to the request.</li> <li>The <code class="highlighter-rouge">?q=$1</code> tells the module how to pass down the parameter. In this case, it’s passed down as the <code class="highlighter-rouge">q</code> parameter.</li> </ul> You can extend this even further by using regular expressions. For example: <div class="highlighter-rouge"><pre class="highlight"><code>RewriteRule ^([^/]*)(.*)$ index.php?first=$1&second=$2 [L,QSA] </code></pre> </div> This will pass down the first part of the path as the <code class="highlighter-rouge">first</code> parameter, and the rest as the <code class="highlighter-rouge">second</code>. So the following request <div class="highlighter-rouge"><pre class="highlight"><code>http://yourhost.com/some/path/somewhere </code></pre> </div> will result in <div class="highlighter-rouge"><pre class="highlight"><code>http://yourhost.com/index.php?first=some&second=path/somewhere </code></pre> </div> This allows for some creative ways to do clean URLs. ###Trouble Shooting If it’s not working, make sure that mod_rewrite is installed on Apache. On a unix system you can just do <div class="highlighter-rouge"><pre class="highlight"><code>sudo a2enmod rewrite </code></pre> </div> to achieve that.

Blog Posts

12 Jan 2015 The Logstash Config Guide
03 Feb 2014 Populate a Symfony 2 Form with the referring entity
03 Jan 2014 MS-SQL Stored Procedures in Sequel: Getting the value of output variables
10 Oct 2013 PHP: The slums of the Programming World?
26 Sep 2013 A DIY Dynamic DNS service using Cloudflare
17 Sep 2013 Setting a default order for Doctrine 2 / Symfony 2
22 Jul 2013 Pre-populating an Entity in a Symfony 2 Form
29 Apr 2013 A Summary of Symfony 2 Authorization
28 Feb 2013 I'm the most Organized Disorganized person you'll meet
08 Jan 2013 PHP's array_merge_recursive function explained
20 Dec 2012 Setting up the Swiftmailer Spooler
09 Jun 2012 Building PHP Projects with Jenkins
17 Mar 2012 Decorator Pattern Implemented Properly In PHP
16 Mar 2012 The Low Hanging Fruit Of Testing
24 Feb 2012 json-encode'ing Private and Protected Properties
28 Jan 2012 22seven.com - Folly, or Genius?
16 Nov 2011 Setting up Puppet on Ubuntu 10.4
05 Oct 2011 PHP Community in South Africa, lack thereof
28 Sep 2011 Why Johnny Died (or, How I Finally Realised Why Singletons And Global Variables Are Bad)
13 Sep 2011 A PHP Virtual Host
18 Jul 2011 A JavaScript trim function
11 Jul 2011 Quick Guide To Creating A Website Quickly
05 May 2011 Good Developers Think like a Startup
08 Feb 2011 Why Object Orientation is Great
11 Oct 2010 Getting Started with node.js
07 Oct 2010 Email Parser for node.js
15 Apr 2010 Why I Code
30 Mar 2010 Twitter API Request Entity too large / HTTP 413 error
26 Mar 2010 How to remove the www from your URL
24 Mar 2010 New look!
18 Mar 2010 SQLite Viewer
18 Mar 2010 Update On Backend
18 Mar 2010 Backend 100 Revisions
18 Mar 2010 My take on single or multiple returns
18 Mar 2010 Firefox requests a page twice?
18 Mar 2010 Parse Http Accept Header
18 Mar 2010 Backend-PHP Framework
18 Mar 2010 Importing a file into a Google Gears DB using PHP
18 Mar 2010 Redirect All Requests To Index.php Using .htaccess

Hire Me

Become more productive by delegating your problem solving to me. I do consulting for clients who need an expert developer, integrator and problem solver.

My core skills are Systems and Web Development in languages such as PHP and Ruby

Contact me for a quote or if you have a question:
jrgns@jrgns.net

A blog about systems and the interwebs

Be sure to check out EagerELK - My efforts to explore the ELK stack.

Talks

I spoke at

Latest Blog Posts

12 Jan 2015 The Logstash Config Guide
03 Feb 2014 Populate a Symfony 2 Form with the referring entity
03 Jan 2014 MS-SQL Stored Procedures in Sequel: Getting the value of output variables
10 Oct 2013 PHP: The slums of the Programming World?
26 Sep 2013 A DIY Dynamic DNS service using Cloudflare
17 Sep 2013 Setting a default order for Doctrine 2 / Symfony 2
22 Jul 2013 Pre-populating an Entity in a Symfony 2 Form
29 Apr 2013 A Summary of Symfony 2 Authorization
28 Feb 2013 I'm the most Organized Disorganized person you'll meet
08 Jan 2013 PHP's array_merge_recursive function explained

More...

Resources

These are some of the books I've read that helped me improve my skills. It might help you as well.

Code Complete

The definitive guide to creating great code. A must read for any developer serious about his craft.

Practical Object-Oriented Design in Ruby: An Agile Primer

The best kind of design is one that gets out of your way. Sandi Metz has a great way of describing such a design.

PHP/Architect's Guide to Enterprise PHP Development

A well structured coder CAN produce quality PHP code. This book includes good advice that will help you get there.

JavaScript Bible

This tome takes you way beyond coding with frameworks such as JQuery, and teaches you how to write proper JavaScript.

JavaScript Web Applications

Go beyond using JavaScript to just pretty up pages and make things move. Create all out web pages using this as your guide.

Feel free to suggest more resources / books in the comments!

Yes, these are affiliate links

blog comments powered by Disqus

Jurgens du Toit {{ site.time | date_to_xmlschema }} http://jrgns.net/ Jurgens du Toit jrgns@jrgns.net {% for post in site.posts %} {{ post.title }} {{ post.date | date_to_xmlschema }} http://jrgns.net{{ post.id }} {{ post.content | xml_escape }} {% endfor %} {% for page in site.pages %} {{ page.title }} http://jrgns.net{{ page.id }} {{ page.content | xml_escape }} {% endfor %}

@import "jekyll-theme-primer";

How CoffeeScript Improved My JavaScript Skills

# How CoffeeScript Improved My JavaScript Skills ## Intro * I write JavaScript for fun. I've played around with Node and JQuery and MooTools and just plain ol' vanilla JavaScript, but I can't call myself a professional JavaScript Coder. * While working with [Dashing](https://github.com/shopify/dashing) I realised that I haven't worked with CoffeeScript yet. Challenge Accepted! * I took an existing module I wrote in JavaScript, and rewrote it in CoffeeScript * It might be comparing apples to oranges, as I didn't set out from the beginning to make this a JS vs CS project. Either way I'll try to be fair. Feel free to call me out if you think I'm missing something or if I'm being prejudiced. * CoffeeScript Coders? Because you want to or have to? ## What I Learnt Or, a quick lesson in how *NOT* to code in JS ## I Suck at JavaScript Prototypes * When you're not doing a lot of JavaScript coding it's easy to forget that JavaScript isn't a "normal" OO language - It's approach to OO is prototypical * When you have a flat object hierarchy, it's very easy to write code like this, and not realise that it's not actually "correct": * You can use that as an object, and even create it with `new`: * But any inheritance will be borked as the properties and methods aren't added to the prototype. ## CoffeeScript generated JS * The prototype is used properly, and you can then easily extend this object into something more specific. ## Build Processes are Awesome * Plain JS doesn't need a build process... * Flat file with all my classes, utilities - It's a mess * But why deny yourself an awesome tool? * CoffeeScript requires at least one build step * So once the work of setting it up is done, get some more goodies * I had one flat file with all my classes, utilities and methods in it. It's a mess. Instead of making the time to understand utilities such as Grunt and Gulp I just mucked on. * CoffeeScript requires at least one build step: compilation. Since I had to do compilation, I started looking into other build tasks allowing me to use multiple files, and voila, my project was better organized. ## Source maps? Source maps! ### Without Source Maps * The code is unrecognizable * Is it your fault or the compilers? * Are the changes I'm making fixing or breaking things? ### With Sourcemaps! * Recognizable code * Direct link between what's happening in the browser and your CoffeeScript * Supported in Chrome and Firefox, FireBUG support is coming ## A few meta-functions go a long way ## Simpler is Better? The Tradeoff ### CoffeeScript... * Hides a lot of boilerplate * Less code (read information) to take in * Easier to look past the code and at the abstraction ### But? * Do you want the extra complexity? * and the extra boilerplate? * what about it being only a subset of JS?

Markdown Demo

# Markdown Demo ## External 1.1 Content 1.1 Note: This will only appear in the speaker notes window. ## External 1.2 Content 1.2 ## External 2 Content 2.1 ## External 3.1 Content 3.1 ## External 3.2 Content 3.2

StaticElastic: An Elasticsearch and Backbone SPA

# StaticElastic: An Elasticsearch and Backbone SPA * Jurgens du Toit / [@jrgns](http://twitter.com/jrgns) * [EagerELK](http://blog.eagerelk.com) # Elasticsearch ![Elasticsearch search trend](http://jrgns.net/talks/static-elastic/img/elasticsearch_trends.png) * Started out as a search engine / interface to Lucene * Started evolving into other use cases * NoSQL? * Elasticsearch (the company) -> Elastic * Currently Version 1.6, active development ----------------------------------------- # Elastic / ELK / Found * Developed by Elastic (valued at > $700 mil) * The rest of the ELK stack * Support Contracts * Plugins - Marvel - Shield - Watcher * Talk about the company - community involvement, etc. ----------------------------------------- ### Vitals ![The statistics](http://jrgns.net/talks/static-elastic/img/vitalstatistix.jpg) * REST interface to a schema-free JSON document store * Plugin Architecture * Near Real Time reads * Search * Aggregations * Basic CRUD ----------------------------------------- ### Architecture ![Architecture](http://jrgns.net/talks/static-elastic/img/architecture.jpeg) * Runs over Lucene and provides distribution and an API * Meant to run in a Cluster * Shards and Replicas provides redundancy * CAP - Consistency, Availability, Partition tolerance * Indices and Types * Mappings ## Backbone * One of the first MVP libraries / frameworks * Lightweight * Easy to parse * Serves as a prototype of how to implement an SPA using ES as a backend ## SPA: Why Backbone and ES ![Rev it up!](http://jrgns.net/talks/static-elastic/img/fastandfurious.jpg) * HTTP, REST and CORS out of the box * Can't serve files like CouchDB, but hey * Versioning * Relations * Routing * Silos * Oh, yes, search! - highlighting - spanning - fuzzy search - more of the same - relational - geo ## Why NOT Elasticsearch ![It's gone!](http://jrgns.net/talks/static-elastic/img/needle-in-a-haystack.jpg">

<% if (page.attributes.highlight) { %> <%= page.attributes.highlight.title %> <% } else { %> <%= page.attributes.title %> <% } %>

<% }) %> ``` ----------------------------------------- ```javascript save: function(key, val, options) { if (typeof key === 'object') { key._version = this.attributes._version; } return Backbone.Model.prototype.save.call(this, key, val, options); }, toJSON: function(options) { var data = _.clone(this.attributes); // Don't send the meta fields to the server when POST / PUT ing _.each(this.meta_fields, function(field) { if (data[field]) { delete data[field]; } }); return data; } ``` ----------------------------------------- ```javascript // Backbone.View.Pages loadResults: function(evt) { var options = { remove: false, processData: this.search === '', data: { from: this.from } }; if (this.search !== '') { options.data.query = { term: { title: this.search } }; options.data.highlight = { pre_tags: [ '' ], post_tags: [ '' ], fields: { 'title': {} } }; options.type = 'POST'; options.data = JSON.stringify(options.data); } Pages.fetch(options); }, ``` ## Resources ![Resources](http://jrgns.net/talks/static-elastic/img/resources.jpg)