Today, I upgraded the memory on my Macbook Pro (2010) from 4 GB to 8 GB. One of the main reasons for the memory upgrade was to be able to run Mircosoft Windows 7 virtual machine using Oracle VirtualBox. However, I noticed that VirtualBox was not able to see more than 4 GB of memory even after the upgrade while the system reported that there was 8 GB of installed memory.

A quick research showed that Mac OS X 10.6 (Snow Leopard) uses 32-bit kernel by default. This limits the applications to use only 4 GB of memory. In order for the applications to use larger memories, one need to use the 64-bit kernel (provided that you are on a 64-bit platform). There is an Apple support page that describes how to select the desired kernel.

I changed the defaults to use 64-bit kernel.

$ sudo systemsetup -setkernelbootarchitecture x86_64

This configuration is stored in the file /Library/Preferences/SystemConfiguration/com.apple.Boot.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>Kernel Architecture</key>
        <string>x86_64</string>
        <key>Kernel Flags</key>
        <string></string>
</dict>
</plist>

After making this change and rebooting, VirtualBox was able to see the whole memory.

GNU Emacs‘s compilation mode is capable of detecting error messages from various standard compilers and build tools. However, it is fairy common for one to run into a format of error message that Emacs can’t handle by default.

As you know, Emacs is highly extensible and it provides compilation-error-regexp-alist for accomplishing this. I have done this a couple of times earlier and I had to do it one more time lately for adding regular expression for Maven error messages. Construction of these regular expressions is not fun for everyone. However, regexp-builder makes it easier to construct the regular expression interactively.

Because the documentation around compilation-error-regexp-alist isn’t very intuitive for most, I have decided to make this screencast that describes how to approach this from scratch. Sorry about the typos! For best quality, watch it in HD.

In summary, here is the customization that has to be added to .emacs.

(add-to-list 'compilation-error-regexp-alist 'maven)
(add-to-list 'compilation-error-regexp-alist-alist
	     '(maven "\\[ERROR\\] \\(.+?\\):\\[\\([0-9]+\\),\\([0-9]+\\)\\].*"
		     1 2 3))

Here is what Emacs’s inbuilt documentation says about compilation-error-regular-expression-alist:

compilation-error-regexp-alist is a variable defined in `compile.el'.
Its value is 
(maven absoft ada aix ant bash borland caml comma edg-1 edg-2 epc ftnchek iar ibm irix java jikes-file jikes-line gnu gcc-include lcc makepp mips-1 mips-2 msft omake oracle perl php rxp sparc-pascal-file sparc-pascal-line sparc-pascal-example sun sun-ada watcom 4bsd gcov-file gcov-header gcov-nomark gcov-called-line gcov-never-called perl--Pod::Checker perl--Test perl--Test2 perl--Test::Harness weblint)
 
 
Documentation:
Alist that specifies how to match errors in compiler output.
On GNU and Unix, any string is a valid filename, so these
matchers must make some common sense assumptions, which catch
normal cases.  A shorter list will be lighter on resource usage.
 
Instead of an alist element, you can use a symbol, which is
looked up in `compilation-error-regexp-alist-alist'.  You can see
the predefined symbols and their effects in the file
`etc/compilation.txt' (linked below if you are customizing this).
 
Each elt has the form (REGEXP FILE [LINE COLUMN TYPE HYPERLINK
HIGHLIGHT...]).  If REGEXP matches, the FILE'th subexpression
gives the file name, and the LINE'th subexpression gives the line
number.  The COLUMN'th subexpression gives the column number on
that line.
 
If FILE, LINE or COLUMN are nil or that index didn't match, that
information is not present on the matched line.  In that case the
file name is assumed to be the same as the previous one in the
buffer, line number defaults to 1 and column defaults to
beginning of line's indentation.
 
FILE can also have the form (FILE FORMAT...), where the FORMATs
(e.g. "%s.c") will be applied in turn to the recognized file
name, until a file of that name is found.  Or FILE can also be a
function that returns (FILENAME) or (RELATIVE-FILENAME . DIRNAME).
In the former case, FILENAME may be relative or absolute.
 
LINE can also be of the form (LINE . END-LINE) meaning a range
of lines.  COLUMN can also be of the form (COLUMN . END-COLUMN)
meaning a range of columns starting on LINE and ending on
END-LINE, if that matched.
 
TYPE is 2 or nil for a real error or 1 for warning or 0 for info.
TYPE can also be of the form (WARNING . INFO).  In that case this
will be equivalent to 1 if the WARNING'th subexpression matched
or else equivalent to 0 if the INFO'th subexpression matched.
See `compilation-error-face', `compilation-warning-face',
`compilation-info-face' and `compilation-skip-threshold'.
 
What matched the HYPERLINK'th subexpression has `mouse-face' and
`compilation-message-face' applied.  If this is nil, the text
matched by the whole REGEXP becomes the hyperlink.
 
Additional HIGHLIGHTs as described under `font-lock-keywords' can
be added.
 
You can customize this variable.

Today, I planned to go back to the basics by taking 6.001 Structure and Interpretation of Computer Programs offered by MIT OpenCourseWare. I’ll save the reason behind it for another post.

For running the programs that are used in the class, I decided to use MIT/GNU Scheme. I am running GNU Emacs 23 on my Mac OS X. After some research, I figured out the best way of doing this is through xscheme.

First, download the MIT/GNU Scheme binary for Mac OS X and copy it to your Applications directory. Then configure Emacs to use the downloaded binary by adding the following lines to your .emacs.

(setq scheme-program-name
      "/Applications/mit-scheme.app/Contents/Resources/mit-scheme")
(require 'xscheme)

Now write your Scheme program.

; Compute the square root of a given number using successive
; approximation.
 
(define (sqrt value)
  (define (is-good-enough? guess value)
    (< (abs (- (* guess guess) value)) 0.0000001))
 
  (define (try guess value)
    (if (is-good-enough? guess value)
	guess
	(try (/ (+ guess (/ value guess)) 2) value)))
 
  (try 1 value))
 
(sqrt 4.0)

Invoke the Scheme process by 'M-x run-scheme'. Send the Scheme buffer to the Scheme process by 'M-o' and now you are able to run Scheme programs from Emacs.

Below is the screenshot of Scheme running under my Emacs session.

MIT Scheme running under GNU Emacs

Recently, I started playing with a lot of Open Source products from Apache Software Foundation. It all started with Hadoop, HBase and Cassandra. Day after day, I am getting my hands dirty on more Apache Foundation’s products like Ant, Maven, Archiva and Thrift.

When trying to build HBase from source, I noticed that the project was using Subversion for version control. I found it quite odd to see a modern project like HBase not using a distributed version control tool like Mercurial or Git. Soon, I realized that all Apache projects’ source code were maintained in Subversion. Then, I made a comment to my co-worker that, “Maybe Apache Foundation took over Subversion too!”, soon to realize that it was true. We learned that Subversion became an Apache Incubator project in 2009 and became an Apache top-level project in 2010.

I am really amazed by the number of projects that are now part of the Apache Software Foundation. Go Apache!

I have been following Test-Driven Development for a few years now. Even though TDD is widespread, often I come across a few friends who aren’t very familiar with TDD approach. It took a while for me to really appreciate TDD since I was introduced to it. When I demonstrated TDD in action, I got a few of my friends interested.

We have our own test framework that we use in our project which was primarily developed by David Carlton. It works very well for our needs. However, for my personal projects, I wanted to try something that is more widely used in the industry. I started using CppUnit for a while until I found Boost Test Library coming a long way. Now, I use Boost Test Library for all my personal projects. It is very easy to setup tests and I really like it.

I also wanted to write a quick introduction to Boost Test Library. So, I thought that I will put down a screencast that will solve two purposes of demonstrating Boost Test Library and serve as an introduction to TDD. This is not an extensive demo or an introduction. I have chosen a really simple problem that is often asked in preliminary rounds of technical interviews. But, it is a good place to start. I don’t guarantee that the solution is efficient. But, it is correct to my knowledge. Please feel free to suggest issues or improvements.

Please note that a HD version of this video is available when viewed on Vimeo’s site.

Problem:
Implement a function ‘int atoi(const std::string &val)‘ that converts the given string in decimal notation to its integer value. Return 0 if the input is not a valid integer.

Source code:

#include <string>
 
#define BOOST_TEST_MODULE atoi
#define BOOST_TEST_DYN_LINK
 
#include <boost/test/unit_test.hpp>
 
namespace impl {
 
int atoi(const std::string &val) {
    int result = 0;
    int multiplier = 1;
    bool negative = false;
 
    for (std::string::const_reverse_iterator i = val.rbegin();
         i < val.rend();  ++i) {
        if (*i <= '9' && *i>= '0' ) {
            result += (*i - '0') * multiplier;
            multiplier *= 10;
        } else {
            if (*i == '-' && i == val.rend() - 1) {
                negative = true;
            } else {
                return 0;
            }
        }
    }
    return negative ? -result : result;
}
 
} // namespace impl
 
BOOST_AUTO_TEST_CASE(unit_position) {
    BOOST_CHECK_EQUAL(impl::atoi("6"), 6);
}
 
BOOST_AUTO_TEST_CASE(tenth_position) {
    BOOST_CHECK_EQUAL(impl::atoi("45"), 45);
}
 
BOOST_AUTO_TEST_CASE(large_number) {
    BOOST_CHECK_EQUAL(impl::atoi("123456789"), 123456789);
}
 
BOOST_AUTO_TEST_CASE(negative_number) {
    BOOST_CHECK_EQUAL(impl::atoi("-876"), -876);
}
 
BOOST_AUTO_TEST_CASE(sign_in_wrong_position) {
    BOOST_CHECK_EQUAL(impl::atoi("72-56"), 0);
}
 
BOOST_AUTO_TEST_CASE(invalid) {
    BOOST_CHECK_EQUAL(impl::atoi("abcd"), 0);
}

Compilation and execution:

# g++ -g -lboost_unit_test_framework -o atoi atoi.cpp && ./atoi --log_level=test_suite --report_level=short

My obsession for GNU Emacs has grown over years to an extent where I managed to get a significant amount of users to adopt Emacs. In the past 10 years, I have learned a lot of nice tricks that I can do on Emacs to improve my productivity. So, I have decided to create a series of screencasts demonstrating some of those.

I will start with a very simple one, macros. Quoting from Emacs documentation, “A keyboard macro is a command defined by an Emacs user to stand for another sequence of keys. For example, if you discover that you are about to type C-n M-d C-d forty times, you can speed your work by defining a keyboard macro to do C-n M-d C-d, and then executing it 39 more times.”

In this demo, I have taken a real world example where you have to add C++ class member variables and accessors. There are other efficient ways to do such tasks in GNU Emacs. I personally use yasnippets to do these things. However, this approach is shown just to demonstrate keyboard macros. To supplement this video, please take a look at the keyboard macro documentation that is available within Emacs.

If you have ever wanted to get the command line options that were passed to a running Solaris process, you might have noticed that the output of command line arguments from ps is truncated to 80 characters. Looking into /usr/include/sys/procfs.h will reveal the reason why! This is because of the restriction in struct psinfo. Here are the relevant fields from the definition of struct psinfo.

#define	PRFNSZ		16	/* Maximum size of execed filename */
#define	PRARGSZ		80	/* number of chars of arguments */
 
typedef struct psinfo {
         /* Fields omitted */
         char pr_fname[PRFNSZ];    /* name of exec'ed file */
         char pr_psargs[PRARGSZ];  /* initial characters of arg list */
         /* Fields omitted */
} psinfo_t;

So, due to the 80 characters restriction in psinfo::pr_psargs, the kernel will not be keeping track of arguments beyond the limit. Now, the only way to get the information is from the process’ memory of argv. In order to do this, you should have access to read the processes’ memory. This is the trick employed by both pargs and BSD version of ps with -ww switch.

To get the full length command line arguments passed to a process, you can do one of the following.

$ /usr/ucb/ps eww <pid>

$ pargs -l <pid>

One catch here is that, if the process has modified the argv since it was started, the output reported by both ps and pargs will show the modified data and not the initial arguments that were passed in. However, modifying argv within a program is not a standard practice and hence the chance of encountering such a scenario is remote.

I had a previous post on how to turn on core files for set-UID, set-GID processes under Linux. Recently we ran into the same problem on Solaris. To turn on core files for set-id processes, use coreadm.

$ pfexec coreadm -e global-setid

Please keep in mind that these core files can have information that non-privileged user isn’t supposed to know. Quoting from Solaris man page:

     A process that is or ever has been setuid  or  setgid  since
     its  last  exec(2)  presents  security issues that relate to
     dumping  core.  Similarly,  a  process  that  initially  had
     superuser  privileges  and  lost  those  privileges  through
     setuid(2) also presents security issues that are related  to
     dumping core. A process of either type can contain sensitive
     information in  its  address  space  to  which  the  current
     nonprivileged  owner  of the process should not have access.
     If setid core files are enabled, they are created  mode  600
     and owned by the superuser.