My tryst with technology

dtrace

2011-02-12T07:58:00.000-08:00

I had written about Mac dtrace introduced in 10.5. Attended a cool lecture yesterday which covered dtrace introduced in Solaris.

dtrace as the name conveys is dynamic tracing. It allows you to trace the kernel as well as user land processes thus filling in the void by other tools like ktrace and the likes.
dtrace doesn't have any additional overhead. Why? Read on. The dtrace framework is composed of providers - wherein providers are the ones which provide the probes for the various components of the kernel/userland susbsystem. When a dtrace command or a D script (or python/perl/korn script which has dtrace commands) is fired, then the providers which are nothing but loadable kernel modules (LKM) are loaded by the dtrace VM, thus enabling the trace for the particular subsystem. This on demand patching of the kernel ensures that there is no overhead, unless enabled - that too only providers that are demanded for, are loaded.

Now how do you load only the providers that you are interested in?
Well, this does take a little practise/playing with the dtrace command, I guess.. Dtracce has a set of providers, each one tackles a particular subsystem. Predicates allow you to drill down to the level that you want for e.g. to a particular app's particular module's tracing - and that too to the particular module's condition of occurence. (the example that i was introduced to by the speaker was app:firefox, method/module:poll, ts>n seconds and so on.)

All and all, sounds like a cool thing to be armed with. (Been there, done that without dtrace - with ktrace, kdb and weeks of debugging.)

To read more on this:
http://www.sun.com/bigadmin/content/dtrace/dtrace_usenix.pdf
http://developers.sun.com/solaris/articles/power_of_dtrace.pdf

Sense of Usability

2010-01-09T08:43:00.000-08:00

I've been fortunate to interact and work with some really good professionals. And one thing common with most of them (if not all) has been that they have excellent sense of usability. Don't know what's the link, but they can recognize and in turn use a good feature to the best effect. They can tell a clean interface from a bad one. They can tell a well designed site from the clutter. What's more, they themselves can be trusted to give ideas for good usability.

From my experiences, I've seen that usability is a very important aspect - be it an API interface or user interface. And some folks manage both with equal brilliance!

Lately, Sharan just reinstated this belief of mine.

Usability is the key. Everywhere. And not just a term to be dismissed off as UE related. It is there in every aspect of programming/programs developed/used et al.

Domain Driven Design

2009-10-19T10:53:00.000-07:00

Recently read a few sections of the book Domain Driven Design. The importance of understanding the domain is not to be undermined.

It is not that you can't design/develop without knowing the domain. And not that you could know the domain overnight and design.

Knowing a domain and designing/refactoring is an ongoing process. And the more you know, the more you improvise.

I, for one, had never given so much of a thought to domain knowledge, but the examples in the book were really enlightening. I could relate to many of them and also realised that in reality knowing the domain had helped me get many things right in the last 6 yrs!

All and all, the design tips and methodologies in the book are uber cool and helpful too.

Updated: The book: http://www.amazon.com/gp/product/0321125215

2009-08-22T02:08:00.000-07:00

Enabling options based on filetype in vim. Picked from here


:au BufRead    *.c     so ~/.vim/c.vim
:au BufNewFile *.c     so ~/.vim/c.vim
:au BufRead    *.py    so ~/.vim/py.vim
:au BufNewFile *.py    so ~/.vim/py.vim

Then put this in ~/.vim/c.vim:

set shiftwidth=8

And this in ~/.vim/py.vim:

set shiftwidth=8

That'll cause vim to source the specified file when the corresponding file type is read (or when a new file of the corresponding type is created).

Recording some good links

2009-07-26T09:56:00.000-07:00

http://www.ddj.com/go-parallel/

http://www.linux-mag.com/cache/7407/1.html

Cilk++

2009-07-24T22:32:00.000-07:00

With the advent of 16, 24 (and also 32?) cored machines, multi-core machines are foraying in a big way. Most applications programmed aren't multi-core aware and hence though multi-core hardware prices aren't daunting, the applications can't really scale to it. Redesigning the legacy apps for multi-core aware is not a small task. However suites like Cilk++ do make it so.

Cilk++ comes with a complete suite of compiler, run time system et al, which enables legacy applications that have been designed with no scaling in sight to be multi-core aware with minimal changes. Believe it or not, your code requires as less change as using three keywords effectively to indicate parallelism and synchronization – cilk_for, clik_spawn (for areas which have recursive calls or the types to make use of the multi-cores), cilk_sync (gather results from all the calls). Cilk provides race detector and reducers (race free global variables) as well.

To know more, read this:

http://www.cilk.com/multicore-products/cilk-solution-overview/

On using the three keywords to indicate parallelism, the Cilk++ compiler compiles the serial code into multi-core enabled parallel application.

N-Queens Problem

A good starting point to understand how this can be used is the N-Queens problem:

http://www.cilk.com/multicore-blog/bid/6381/Multicore-enabling-the-N-Queens-Problem-Using-Cilk.

There are other problems discussed as well. You could track the cilk blog to know more.

And to know more about the Cilk language itself:

http://supertech.csail.mit.edu/papers/cilk5.pdf

Python & Scalability

2008-10-28T01:45:00.000-07:00

Was checking on Python scalability.

Found some good links
http://highscalability.com/youtube-architecture
http://highscalability.com/tags/python
http://www.python.org/about/success/strakt/ (This one uses Twisted as well)

Generally you can check here: http://www.python.org/about/success/

Couldn't resist this one: http://www.python.org/about/success/rackspace/

And yes, python is not only a scripting language, it is an object oriented programming language too!

A pat for Twisted is here as well:
http://news.zdnet.co.uk/software/0,1000000121,39246546,00.htm

In search for an ORM

2008-09-14T00:34:00.000-07:00

Sharan has been searching for an ORM for C++. Though we have been able to find many for Java, python etc; couldn't find one which is popular for C++.

A few that I found, though we yet to evaluate are:
http://soci.sourceforge.net/
http://sourceforge.net/projects/open-orm/
Database Template Library: http://dtemplatelib.sourceforge.net/

Found this one mentioned at a couple of places; but the page just refuses to open up
http://ceamus.com/objbuilder/

If anyone has any better suggestions, let me/sharan know.

wake_me_up or wake_me_not

2008-02-22T23:26:00.000-08:00

We had a really tough problem to solve in the last few weeks and we finally got through it.. Every experience of solving a tough problem ends with sweet little learnings. And this time it was about "wake_up" functions on Linux. These functions don't seem to have good documentation anywhere and knowing the nuances of the different variants demands a lot of googling and reading :)
So was quite happy with this find, wherein a person has documented his learnings in one place.

http://groups.google.co.in/group/comp.os.linux.development.system/browse_
thread/thread/607517fcb4ecf29d/d752bf66a0a7bb48?lnk=st&q=wake_up_
interruptible+wake_up_interruptible_all#d752bf66a0a7bb48

And the equivalents of Mac can be found at
http://developer.apple.com/documentation/Darwin/Conceptual/KernelProgramming
/services/chapter_16_section_5.html#//apple_ref/doc/uid/TP30000905-CH219-CHDIDIDF

Hope this will help others in pursuit of wake_up problems!

Macbook Air

2008-01-19T23:34:00.000-08:00

Was reading reviews about MacBook Air and came across some very interesting one-liners on it

"It’s an expensive, disposable toy"
"Apple: Macbook Air inspires laptop Anorexia"
"Steve made a machine for himself, as ever."

Read some interesting stuff at
http://cultofmac.com/

What does GM stand for?

2008-01-19T22:04:00.000-08:00

External beta testing is usually followed by GM build testing and subsequent release. So what is GM in the beta parlance? My project had been in the GM phase for quite a time. However I didn't bother to google for it throughout. Yesterday while reading an interview of an indie developer (Read Sweater), I read about Leopard's Golden Master not being released to Mac developers for testing before official release! And it was then, that I realised what the acronym GM meant. To confirm, I googled for the term which explained:

GM, or Gold Master, is what software is called when it has completed all phases of testing and is ready to be released to the general public

Ref:
http://beta.aol.com/projects.php?project=infocenter&loc=betaglossary#G12

Pitfalls of using threads

2007-12-23T03:08:00.000-08:00

livejournal's cpp community had a post on multi threading with c++, which got me reading on this.

The pitfalls with multi threading
- thread safe functions/non re-entrant functions
- deadlock
- race code
- pitfalls seen especially with C++ code (making initialization routines thread safe etc)

Boost.Threads (a little outdated article per the issues discussed still are a part of multithreading environment)
http://www.ddj.com/cpp/184401518

http://www.boost.org/doc/html/thread.html#id1710137

Pthreads
http://www.yolinux.com/TUTORIALS/LinuxTutorialPosixThreads.html

Testing and debugging is highly non-deterministic in a threaded environment, which adds to the difficulties.

Read to know more!

XCode 3.0 Instruments

2007-12-17T19:56:00.000-08:00

XCode 3.0 has some exciting stuff introduced- Instruments. Read more on this:
http://www.apple.com/macosx/developertools/instruments.html

Mac OS 10.5

2007-12-17T18:56:00.000-08:00

I have been searching for information on Mac Os 10.5. Although there is a lot of scattered information, there is not a single site which gives an overview (technically) of the changes. There is a lot of information on the product/user experience stuff, but hardly any take on the development aspects.

OS 10.5 with 64-bit compatibility: This is pretty encouraging to read. One doesn't need to have any driver/device changes for 64-bit. And 32-bit applications work smoothly too. Also, it follows LP64 model, so Unix 64-bit apps can be ported easily.
http://www.apple.com/macosx/technology/64bit.html

Talking about Unix, OS 10.5 claims to be Unix certified. So any/all unix apps will run on Mac 10.5 as-is, without "any" change!
http://www.apple.com/macosx/technology/unix.html

File system event APIs.. I almost thought this could be used as Kauth is used, before I read it completely. Nevertheless a good feature:
http://developer.apple.com/documentation/Darwin/Conceptual/FSEvents_ProgGuide/TechnologyOverview/chapter_3_section_1.html#//apple_ref/doc/uid/TP40005289-CH3-SW1

However, I found kernel queues, which provide notifications for most events like Kauth does! Need to see, how far we can go with this.. also whether it caters to our purpose better than Kauth. One thing that I see as a plus here is that it gives notification for exit, fork as well.

http://developer.apple.com/documentation/Darwin/Conceptual/FSEvents_ProgGuide/KernelQueues/chapter_6_section_1.html#//apple_ref/doc/uid/TP40005289-CH5-SW2

This link I found, was a little less user oriented than others:
http://www.informationweek.com/news/showArticle.jhtml?articleID=202802284&pgno=1&queryText=

OS 10.5 as a server?
http://www.informationweek.com/news/showArticle.jhtml?articleID=202801226&pgno=1&queryText=

Till, I find some more information on this, Ciao!

DTrace on Mac

2007-12-09T11:02:00.000-08:00

In my search for some help on tracing calls, I found this interesting link:
http://www.mactech.com/articles/mactech/Vol.23/23.11/ExploringLeopardwithDTrace/index.html

ktrace for some reason, isn't providing us the information needed for tracing the shutdown call. Attaching to the process using debugger also resulted into nowhere as the stacks were too many to analyze and we couldn't really nail down the problem.

Some interesting snippets of this link:

You interact with DTrace by writing small programs in the D programming language. These D programs can be saved in text files and run like shell scripts, or they can stretched out right on the command line for quick, ad-hoc use (or if you simply want to impress your friends). An example D script that totals all the system calls made for each process on the system is shown in Listing 1.

Listing 1: syscalls_per_proc.d

Totals up all the system calls made for each process

syscall:::entry
{
  @[execname] = count();
}

.
.
.

And this:

File activity

It can be enlightening to see which files are accessed on a system. For example, you may see that Foo.app is frequently writing to some file, or maybe that Bar.app is calling stat(2) on a log file every 10ms. This information can help you debug your own programs, or perhaps better understand the system in general. Below we use a small D script to print out the name of each file as it's opened.

$ sudo dtrace -s /dev/stdin
syscall::open*:entry
{
  printf("%s %s", execname, copyinstr(arg0));
}
^D
dtrace: script '/dev/stdin' matched 3 probes
CPU     ID      FUNCTION:NAME
 0  17584      open:entry Finder /.vol/234881026/562669
 0  17584      open:entry Finder /.vol/234881026/562669
 1  17584      open:entry iChatAgent /Users/jgm/Library/Caches/...
 0  17584      open:entry iChatAgent /Users/jgm/Library/Caches/...
 1  17584      open:entry iChat /System/Library/PrivateFrameworks/...


^C

This script sets a probe at the entry to all system calls having names beginning with "open". DTrace tells us that our probe description matched three probes. They are: open, open_extended, and open_nocancel. Our action statement prints out the name of the process (execname) that caused the probe to fire, and the first argument (arg0) to the function that matched the probe. Notice that we need to use the copyinstr function here rather than just printing arg0 directly. This is because D scripts execute in the kernel's address space, but the pathname argument to open is stored in user space. We could also modify our D script so that it shows us which files are accessed most often, as follows.

$ sudo dtrace -s /dev/stdin syscall::open*:entry { @[copyinstr(arg0)] = count(); }

...

Although Apple provides no documentation for this, Sun systems has sufficient docs on this.

And oh yes, this article is also courtesy - Greg Miller.

Mac World

2007-12-09T10:33:00.000-08:00

When we initially set out to work on Mac, we Greg Miller's posts helped us a lot. Here are a few links to his blog which are useful for a newbie:

Command line building on KEXT
http://unixjunkie.blogspot.com/2006/12/kernel-extension-by-hand.html

Understanding lists and queues. Vishal would have loved to read this, though getting to understand it himself, must have been more satisfying!
http://unixjunkie.blogspot.com/2006/12/lists-and-queues.html

Sachin had a hard time figuring out a solution for this problem of width that ps assumes. A quick look at the reason for this strange behavior.
http://unixjunkie.blogspot.com/2006/03/strange-difference-in-ps-output.html

Mac UI musts - Command line processing of Cocoa:
http://unixjunkie.blogspot.com/2006/07/command-line-processing-in-cocoa.html

Read more on unixjunkie to know more about small nitty gritties that help a Mac newbie!

Better Explained

2007-12-09T10:05:00.000-08:00

A must read for all - betterexplained.com. Truly, all the concepts here are explained with such lucidness that every word written here is worth its weight in gold!

Do read it, for cool explanations on interesting topics.

Reason for this blog

2007-12-09T09:27:00.000-08:00

I learn a thing or two in my chosen area of work, almost daily. But somewhere down the line, the learning gets lost. This blog is an attempt to record my daily learnings (ok.. not that frequently :)) so that I can quickly retrieve it when needed and also to share my learnings (while earning) with others!

Hope to see a lot of posts from me!