Ting 이네 Software 이야기

This is a kind of reminder.
(On Android)

Source code.

linux/input.h
    struct input_event;
: write 'struct input_event' to input event node.
ex.
    struct input_event ev;
    ...
    write(fd, &ev, sizeof(ev));
    ...

File system

/dev/input/*, /proc/bus/input/*

Touch event 
 

Key event

Sometimes, we want to access framebuffer directly.
Here is template and simple description to do this (on Android  as an example.)
(Note: This is just template. Some modification may be required according to driver.)
(Below way works on Android 2.3 ~ 4.1 emulator by turning off 'Use Host GPU' option at AVD Manager.)

Things used in this example.
(Refer kernel source code for details - comments in code.)
linux/fb.h
    - struct fb_var_screeninfo
        xres, yres, xres_virtual, yres_virtual, xoffset, yoffset, bits_per_pixel
    - struct fb_fix_screeninfo
        smem_len
    - FBIOGET_FSCREENINFO, FBIOGET_VSCREENINFO, FBIOPUT_VSCREENINFO
    - FB_ACTIVATE_NOW, FB_ACTIVATE_FORCE
See fbmem.c as your starting point of source analysis.

...
int                      fd;
struct fb_var_screeninfo vi;
struct fb_fix_screeninfo fi;
void*                    bits;
int                      bpp;    /* byte per pixel */
int                      stride; /* size of stride in pixel */

...
/* Open framebuffer */
if(0 > (fd = open("/dev/graphics/fb0", O_RDWR)) {
    printf("Fail to open fb\n");
    return -1;
}

/* Get fixed information */
if(0 > ioctl(fd, FBIOGET_FSCREENINFO, &fi)) {
    printf("Fail to get fixed info\n")
    return -1;
}

/* Get variable information */
if(0 > ioctl(fd, FBIOGET_VSCREENINFO, &vi)) {
    printf("Failed to get variable info\n");
    return -1;
}

/* Get raw bits buffer */
if(MAP_FAILED == (bits = mmap(0, fi.smem_len,
                              PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0))) {
    printf("Failed to mmap fb\n");
    return -1;
}

/* Calculate useful information */
bpp = vi.bits_per_pixel >> 3;
stride = fi.line_length / bpp;

...
/* Getting raw-image snapshot of current framebuffer */
void* curbits; /* current framebuffer raw data */
curbits = (unsigned char*)bits + (vi.xoffset + vi.yoffset*vi.xres_virtual)*bpp;
memcpy(raw_image_buffer, curbits, vi.yres*stride*bpp);

...
/* Modifying directly */
do_something(curbits...); /* change buffer directly... */

/* Refresh buffer manually */
vi.activate |= FB_ACTIVATE_NOW | FB_ACTIVATE_FORCE;
if(0 > ioctl(fd, FBIOPUT_VSCREENINFO, &vi)) {
    printf("Failed to refresh\n");
    return -1;
}

Linux usually use 'double buffer'.
For this, usually,

vi.yres_virtual == vi.yres * 2
vi.yoffset == 0 or vi.yres
(0 for 1st buffer, vi.yres for 2nd buffer)

But, it's totally dependent on driver. So, to get portability, we should not assume those.

alacarte의 한계를 넘어설 수 있다.
(이하는 Ubuntu 10.10에서의 내용이다.)

아래의 장소들과 깊은 관계되어 있다.
System default value가 들어 있는 곳 :

* /etc/xdg/ -> menus

User value가 들어 있는 곳

* ~/.config/ -> menus
 * ~/.local/share/ -> desktop-directories(- Directory), applications(- Menu Items)

.desktop 들어 있는 standard location

* /usr/share/applications/

크게 '.directory'와 '.desktop' 두 개의 확장자를 기억해 둘 필요가 있다.
'.directory'는 메뉴의 directory정보이고, '.desktop'은 메뉴 item의 정보라는 것 정도만 기억하자.

실험적으로, alacarte로 directory와 item을 만들어 보자.

pannel -> Application -> right click -> Edit Menus

로 들어가면 쉽게 만들어 볼 수 있다. GUI환경이라 skip...

만들었으면,  "~/.local/share/desktop-directories"로 가 보자. 그러면,  alacarte-madexx.directory 라는 file들이 새로 생겼을 것이고, 이 파일을 열어보면, GUI환경에서 새로 추가한 directory정보들과 일치함을 알 수 있다.
그리고 이번에는 "~/.local/share/applications"로 가 보자. alacarte-madexx.desktop 이라는 새로운 파일들을 볼 수 있고, 내용은 GUI환경에서 추가한, menu item정보임을 쉽게 알 수 있다.
이제 "~/.config/menus/"로 가서 applications.menu를 열어보자. 그러면, 위에서 봤던, '.directory' 파일과  '.desktop'파일의 이름이 나오는 메뉴 tag가 새로이 생성된 것을 알 수 있다.
간단히 정리하면, alacarte로 새로운 메뉴와 메뉴 item을 만들면 다음과 같은 파일이 생성/수정 된다.
(alacarte로 첫 메뉴/item을 만들었을 경우를 가정.)

* ~/.local/share/desktop-directories/alacarte-made.directory (추가) -- *1
* ~/.local/share/applications/alacarte-made.desktop (추가) -- *2
* ~/.config/menus/applications.menu (수정) -- *3

*1, *2는 메뉴/item 의 내용에 대한 정보이고, *3이 메뉴 구조에 대한 정보이다.

이제 case study로 실습해보자.
Ubuntu 10.10에서 Ubuntu Software Center에서 wine을 설치하고 이를 Applications 메인 메뉴에 등록해보자.
설치는 간단하니 skip하고...
설치가 끝나면 "/usr/share/applications"에  'wine.desktop, wine-xxxx.desktop' file들이 새롭게 생긴 것을 볼 수 있다. 이제 우리는 이걸 main menu에 잘 등록해 주기만 하면 된다.
그냥 하나씩 등록해도 되겠으나, 일반적으로 wine의 메뉴는 아래의 구조를 가진다.

wine
 +- Programs
 |    +- Accessories
 |        +- Notepad
 |
 |- Browse C: Drive
 |- Configure Wine
 +- Uninstall Wine Software

"각 '.desktop' 파일을 열어보면, category항목들이 보일 것이다. 특히 'wine-notepad.desktop'을 모면 "Categories=Wine-Programs-Accessories;"를 볼 수 있다.  즉 Category구조가 메뉴의 구조가 되고 있다. 이런 사항을 반영해서, *3 를 수정하면 된다.
그냥 direct로 수정해도 되겠으나, 차후 maintenance를 위해서 wine 메뉴는 따로 빼서 만들고 이를 *3에 merge하는 방식이 더 나아 보인다.
아래는 필자가 manual하게 추가한 결과이다.

*3에 추가

<Menu>
  <Name>Wine</Name>
  <MergeFile>wine.menu</MergeFile>
  <Directory>Wine.directory</Directory>
  <Include>
    <Category>Wine</Category>
    <Filename>wine.desktop</Filename>
  </Include>
  <Exclude>
    <Filename>wine.desktop</Filename>
  </Exclude>
  <AppDir>/home/hbg683/.local/share/applications</AppDir>
</Menu>

~/.config/menu/wine.menu 추가

<Menu>
  <Name>Wine</Name>
  <Directory>Wine.directory</Directory>
  <Include>
    <Category>Wine</Category>
     <Filename>wine.desktop</Filename>
  </Include>
  <AppDir>/home/hbg683/.local/share/applications</AppDir>
  <Menu>
    <Name>Programs</Name>
    <Directory>Wine-Program.directory</Directory>
    <DefaultLayout inline="false"/>
    <Menu>
      <Name>Accessories</Name>
      <Directory>Wine-Programs.Accessories.directory</Directory>
      <Include>
        <Category>Wine-Programs-Accessories</Category>
      </Include>
    </Menu>
    <DirectoryDir>/home/hbg683/.local/share/desktop-directories</DirectoryDir>
  </Menu>
  <DefaultLayout inline="false"/>
  <Exclude>
    <Filename>wine.desktop</Filename>
  </Exclude>
</Menu>

'~/.local/share/desktop-directories'에 각종 '.directory'파일 추가

Wine.directory

[Desktop Entry]
Name=Wine
Comment=Wine Windoes Program Loader
Type=Directory
Icon=wine
X-Ubuntu-Gettext-Domain=gnome-menus

Wine-Programs.directory, Wine-Programs-Accessories.directory 추가 (내용은 생략)

끝.

What is stdout, stdin and stderr?
Actually, I am using these without deep understanding.
Now, it's time to deep-dive to it.

Let me focus on stdout. (other twos are similar.)
What type of file default stdout is? Is it normal file? No, definitely. It's device file.
So, two different processes can write data to same stdout directly.
(Normal file doesn't allow this! )
I can easily know which device is used as stdout by checking devices.
(By entering following command in console.

ls -al /dev | grep stdout
    -> stdout -> /proc/self/fd/1

Interesting isn't it?
Let me move forward.

ls -al /proc/self/fd/1
    -> 1 -> /dev/pts/7

Right!.
Even if every process access standard output device with same name 'stdout', it branches to appropriate devices by symbolic link - '/proc/self' is used.
Now I can guess what redirecting stdout is. Let me check it.

# redirecting stdout with sample program.
# something like this.
# main() { for(int i=0; i<10000000; i++) { sleep(1); printf("+"); fflush(stdout); }}
$ ./a.out > t &
$ ls -al /proc/<child pid>/fd/1
    -> 1 -> /xxxx/xxx.../t <= path of target file 't'

Done!

Terms Definition :

Reboot : Real reboot. System restarts from 'boot logo'.
Soft reboot : System restarts from 'Power on animation'. (NOT from 'Boot logo'.)

In user mode

On Dalvik

ANR (Application Not Responding)

* Event dispatching timeout : 5 sec.
=> Windows can't dispatch events over 5 sec.
* Broadcast timeout : 10 sec.
=> Broadcast receiver can't complete things to do in 10 sec.
* Service timeout : 20 sec.
=> Service can't finish jobs in 20 sec..

Hang up

* There is no UI response, even if window dispatched events.
* Framework misses window that events should be passed to. (It doesn't never happens.)
=> Note! : We should wait over 5 sec. to tell it's hang-up or ANR. (See 'Event dispatching timeout')

FC (Force Close)

* Unexpected stop. Ex. null object access,  exceptions.

return from 'main' function.

* It's normal end of process if it is intentional. But in some cases, we may come across unexpected return from 'main'. System's point of view, it's just normal exit of process. So, no error message is left.

At native

Exception (ex. data abort etc)

* Kernel sends appropriate Signal (ex. SIGSEGV) to user process. And user process may be killed. In this case, process is killed due to signal from kernel.  So, we cannot expect UI feedback, but low level logs - ex. tombstone.

In privilege mode

Exception : Reboot. (Kernel 'oops' is executed before reboot. So, we may  be able to find corresponding kernel log/panic data.

Other important and notable cases.

system_process :

* Unexpected stop of  this process may cause system-soft-reboot. (This should be distinguished from reboot caused by kernel exception)

home / launcher

* Unexpected stop causes one-time-flickering of home screen to restart home. Let's imagine following scenario.
-> Home enters unexpected busy state(cannot dispatch user input during this busy time). After 3~4 sec., there is exception and home is stopped -> restarted.
In this case, it may look like just 'Smooth Reset'!

It's very-well-known tip for cyclic queue!
Here is pseudo code!
(Even if this is very -well-known and not difficult to think of, it's worth to remind.)

TYPE Q
    item[SZ] // queue array
    i        // current index
...

FUNC addQ (q, item)
// This is Naive way.
    item[q.i] = item
    if (q.i >= SZ) than q.i = 0

// This is well-known way
    item[q.i] = item
    q.i &= SZ-1 // For this, SZ should be (2^n (n>0))

Here is good article about this!

http://www.linuxprogrammingblog.com/threads-and-fork-think-twice-before-using-them

First of all, I am going to avoid way that using popular tools used in desktop Linux, because in embedded environment(embedded Linux like Android), we cannot expect them.
Before talking about memory analysis, let's look over fundamental concepts related with memory.
(This article is written based on Kernel 2.6.29)

There are 4 type of memory.
private clean, private dirty, shared clean and shared dirty are those.
* Clean vs. Dirty.
Clean means "It doesn't affect to system in point of semantics." So, we can abandon this at any time. Usually, mmap()ed or unwritten memory can be it.
Dirty is exactly opposite.
* Private vs. Shared
This is trivial.

Here is example in Android,
Shared clean : common dex files.
Private clean : application specific dex files
Shared dirty : library "live" dex structures(ex. Class objects), shared copy-on-write heap. - That's why 'Zygote' exists.
Private dirty : application "live" dex structures, application heap.

Usually, clean memory is not interesting subject.
Most memory analysis is focused on dirty memory especially private dirty.
(shared dirty is also important in some cases.)

Linux uses Virtual Memory(henceforth VM). I think reader already familiar with this. Let's move one step forward. Usually, "demand paging" is used. By using "demand paging", Linux doesn't use RAM space before the page is really requested. Then what this exactly means. Let's see below codes.

#define _BUFSZ (1024*1024*10)
static int _mem[_BUFSZ];
int main (int argc, char* argv[]) {
    int i;
    /* --- (*1) --- */
    for(i=0; i<_BUFSZ; i++) {
        _mem[i] = i;
    }
    /* --- (*2) --- */
}

As you see, "sizeof(_mem)" is sizeof(int)*10*1024*1024 = 40MB (let's assume that sizeof(int)==4).
But, at (*1), _mem is not REALLY requested yet. So, Linux doesn't allocate pages in the RAM. But, at (*2), _mem is requested. So, pages for _mem is in RAM.
OK? Later, we will confirm this from the Kernel.

Now, let's go to the practical stage.
As reader already may know, there is special file system - called procfs - in Linux. We can get lots of kernel information from procfs including memory information.
Try "cat /proc/meminfo".
Then you can see lots of information about memory. Let's ignore all others except for 'MemTotal', 'MemFree', 'Buffers', 'Cached'
(Documents in Kernel source are quoted for below description)
-----------------------------------------------
MemTotal : Total usable ram (i.e. physical ram minus a few reserved bits and the kernel binary code)
MemFree: The sum of LowFree + HighFree
LowFree: Lowmem is memory which can be used for everything that highmem can be used for, but it is also available for the kernel's use for its own data structures.  Among many other things, it is where everything from the Slab is allocated.  Bad things happen when you're out of lowmem.
HighFree: Highmem is all memory above ~860MB of physical memory Highmem areas are for use by userspace programs, or for the pagecache.  The kernel must use tricks to access this memory, making it slower to access than lowmem.
Buffers: Relatively temporary storage for raw disk blocks shouldn't get tremendously large (20MB or so)
Cached: in-memory cache for files read from the disk (the pagecache). Doesn't include SwapCached.
-----------------------------------------------

Now, we know that size of total memory and free memory etc.
Type 'adb shell ps'
we can see VSIZE(henceforth VSS), RSS(Resident Set Size) column. VSS is amount of memory that process requires. RSS is amount of memory that is REALLY located at physical memory - demanded one!.
As mentioned above, main reason of difference between VSS and RSS  is 'demand paging'.
Now, let's sum all RSSs. Interestingly sum of RSSs is larger than total memory size from 'meminfo'
Why? Can you guess? Right. Due to shared one. For example, In case of Android, there are some prelinked objects. And those are shared among processes. And process RSS size includes those shared one. That's why sum of RSSs is larger than memory size.

To understand deeper about VSS, see following example.
Make empty program, execute it and check it's VSS. For example

void main() { sleep(1000); }

It's size is over 1M!. Why? Kernel reserves memory blocks to handle process itself - for example, page table, control block etc.  As an example, in case of page table, normal 32-bit machine uses 4Kb page and 4G virtual memory. So, number of pages are 4G/4K = 1M. To keep tracking 1M pages, definitely, certain amount of memory is required.
So, at least some - actually, not small - amount of memory is required even in very tiny process.

As mentioned above RSS includes shared memory block. But, we want to know reasonable size of memory that is really located in.
Here is PSS(Proportional Set Size).

PSS = "Non-shared process private memory" + "Shared memory" / "Number of processes that shares those".

Great!. So, sum of PSS is real size of occupied memory.
Then, how can we know it?
The most primitive way is checking

/proc/<PID>/smaps

You can easily found PSS field in it. (For more details, see kernel source code 'task_mmu.c')

smaps also shows memory usage of each memory type for each Virtual Memory Area(VMA).
So, we can analyse memory status deeply through smaps (usually, focusing on private dirty).

Let's deep dive to memory world more.
We can easily guess and know followings.
- local global, static, heap (usually allocated by malloc & new etc) memories are all private. And those are clean until something is written on them.
- memory used by mmap is shared and can be clean and dirty. But mapped memory is also on-demand. What does this mean? Let's assume that 4 processes share one anonymous mmaped area. At first, they don't access to the memory. So, RSS/PSS regarding this area is '0'. But after process 1 accesses to the memory, this process's RSS/PSS is increased. And then, when process 2 accesses to the memory, this process's RSS/PSS is increased. But, in this case, memory is shared by two processes (1, 2). So, amount of memory increased in terms of PSS is half of amount increased in terms of RSS.

Here is example of this case.
Code for test process looks like this.

#include <stdio.h>
#include <stdint.h>
#include <stdlib.h>
#include <time.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/mman.h>
#include <memory.h>

#define MEMSZ 1024 * 1024 * 10
#define MAPSZ 1024 * 1024 * 6

int
main(int argc, const char *argv[]) {
	char  pidbuf[32];
	int   fd, cnt, done;
	char *b, *map;

	b = malloc(MEMSZ);
	map = (char *)mmap(NULL,
			   MAPSZ,
			   PROT_READ | PROT_WRITE,
			   MAP_ANON | MAP_SHARED,
			   -1,
			   0);
	done = 0;
	cnt = 3;
	while (cnt--) {
		switch (fork()) {
		case -1:
			printf("ERR fork\n");
			exit(0);

		case 0: /* child */
			done = 1;
		}
		if (done)
			break;
	}

	while (1) {
		if (0 <= open("done", O_RDONLY))
			break;

		snprintf(pidbuf, sizeof(pidbuf), "%d-mem", getpid());
		if (0 <= (fd = open(pidbuf, O_RDONLY))) {
			close(fd);
			unlink(pidbuf);
			/* access on demand */
			memset(b, 0, MEMSZ);
		}

		snprintf(pidbuf, sizeof(pidbuf), "%d-map", getpid());
		if (0 <= (fd = open(pidbuf, O_RDONLY))) {
			char h;
			int  sz;
			close(fd);
			unlink(pidbuf);
			sz = MAPSZ / 2;
			/* access MAPSZ / 2 -> read on demand */
			while (sz--)
				h ^= map[sz];
#ifdef WRITE_TEST
			sz = MAPSZ / 2;
			while (sz--)
				map[sz] ^= map[sz];
#endif

		}

		sleep(1);
	}
	return EXIT_SUCCESS;
}
 

Followings are smap report regarding mmapped memory area.

[ Original ]
40a93000-41093000 rw-s 00000000 00:07 4270       /dev/zero (deleted)
Size:               6144 kB
Rss:                   0 kB
Pss:                   0 kB
Shared_Clean:          0 kB
Shared_Dirty:          0 kB
Private_Clean:         0 kB
Private_Dirty:         0 kB
Referenced:            0 kB
Swap:                  0 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB


[After read mmapped area by creating "-map" file]
40a93000-41093000 rw-s 00000000 00:07 4270       /dev/zero (deleted)
Size:               6144 kB
Rss:                3072 kB
Pss:                3072 kB
Shared_Clean:          0 kB
Shared_Dirty:          0 kB
Private_Clean:      3072 kB
Private_Dirty:         0 kB
Referenced:         3072 kB
Swap:                  0 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB

[After read mmapped area by creating "-map" file for another child process forking from same parent]
40a93000-41093000 rw-s 00000000 00:07 4270       /dev/zero (deleted)
Size:               6144 kB
Rss:                3072 kB
Pss:                1536 kB
Shared_Clean:       3072 kB
Shared_Dirty:          0 kB
Private_Clean:         0 kB
Private_Dirty:         0 kB
Referenced:         3072 kB
Swap:                  0 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB 

Please take your attention to change of RSS/PSS memory size.
And, one more interesting point is, memory is still shared clean because operation is just 'read'.

Then, what happen if mmap with MAP_PRIVATE instead of MAP_SHARED.
In this case, memory allocated by mmap is handled just like memory allocated by malloc.
So, with high probability, two memory area are merged int to one. And you may see one private memory area whose size is 16M.

Next topic is very interesting.
Let's try with MAP_PRIVATE.

7f987c7aa000-7f987d7ab000 rw-p 00000000 00:00 0
Size:              16388 kB
Rss:                   4 kB
Pss:                   1 kB
Shared_Clean:          0 kB
Shared_Dirty:          4 kB
Private_Clean:         0 kB
Private_Dirty:         0 kB
Referenced:            4 kB
Anonymous:             4 kB
AnonHugePages:         0 kB
Swap:                  0 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Locked:                0 kB
In case of read mmapped area, memory is not allocated even if read operation is executed.
(This is different with the case of MAP_SHARED.)
Why? Because MAP_PRIVATE maps memory with copy-on-write.
So, just reading don't need to allocate memory.
Let's try with enable 'WRITE_TEST' define switch.
7f001b786000-7f001c787000 rw-p 00000000 00:00 0 
Size:              16388 kB
Rss:                3076 kB
Pss:                3073 kB
Shared_Clean:          0 kB
Shared_Dirty:          4 kB
Private_Clean:         0 kB
Private_Dirty:      3072 kB
Referenced:         3076 kB
Anonymous:          3076 kB
AnonHugePages:         0 kB
Swap:                  0 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Locked:                0 kB
As you can see, memory is allocated successfully as 'Private Dirty'.
As next step let's see writing mapped area by it's child.
[After write mmapped area by creating "-map" file for another child process forking from same parent]
7f001b786000-7f001c787000 rw-p 00000000 00:00 0 
Size:              16388 kB
Rss:                3076 kB
Pss:                3073 kB
Shared_Clean:          0 kB
Shared_Dirty:          4 kB
Private_Clean:         0 kB
Private_Dirty:      3072 kB
Referenced:         3076 kB
Anonymous:          3076 kB
AnonHugePages:         0 kB
Swap:                  0 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Locked:                0 kB
PSS and RSS is unchanged.
This is what we expected

But, there is interesting case.
Let's see below vma information
b5a7f000-b5f7f000 -w-p 00000000 00:04 24098      xxxxxxxxxxxxxxx
Size:               5120 kB
Rss:                5120 kB
Pss:                1280 kB
Shared_Clean:          0 kB
Shared_Dirty:       5120 kB
Private_Clean:         0 kB
Private_Dirty:         0 kB
Referenced:            0 kB
Anonymous:          5120 kB
AnonHugePages:         0 kB
Swap:                  0 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Locked:                0 kB 
Even if memory area is private and writable, RSS != PSS - That is, it is shared!
In case of read-only private area, it can be shared - ex. loaded shared library code.
But, this is writable private area!
What happened? <= <TODO>I need to investiage more about it!!!

kenel uses VM_MAYSHARE flag to tell this is 'p' or 's' - see task_mmu.c in kernel.
I'm not sure that VM_MAYSHARE is more valuable information then VM_SHARED.
But, I have to analyse this case deeper... (to be updated after more analysis...)

Next case is ashmem.
memory mapped with MAP_PRIVATE, can be shared in case of ashmem.
You can test this case by using below code in Android (NDK).

#include <stdio.h>
#include <stdint.h>
#include <stdlib.h>
#include <time.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/mman.h>
#include <memory.h>
#include <sys/ioctl.h>
#include <linux limits.h>
#include <linux ioctl.h>


#define ASHMEM_NAME_LEN		256
#define ASHMEM_NAME_DEF		"dev/ashmem"

/* Return values from ASHMEM_PIN: Was the mapping purged while unpinned? */
#define ASHMEM_NOT_REAPED	0
#define ASHMEM_WAS_REAPED	1

/* Return values from ASHMEM_UNPIN: Is the mapping now pinned or unpinned? */
#define ASHMEM_NOW_UNPINNED	0
#define ASHMEM_NOW_PINNED	1

#define __ASHMEMIOC		0x77

#define ASHMEM_SET_NAME		_IOW(__ASHMEMIOC, 1, char[ASHMEM_NAME_LEN])
#define ASHMEM_GET_NAME		_IOR(__ASHMEMIOC, 2, char[ASHMEM_NAME_LEN])
#define ASHMEM_SET_SIZE		_IOW(__ASHMEMIOC, 3, size_t)
#define ASHMEM_GET_SIZE		_IO(__ASHMEMIOC, 4)
#define ASHMEM_SET_PROT_MASK	_IOW(__ASHMEMIOC, 5, unsigned long)
#define ASHMEM_GET_PROT_MASK	_IO(__ASHMEMIOC, 6)
#define ASHMEM_PIN		_IO(__ASHMEMIOC, 7)
#define ASHMEM_UNPIN		_IO(__ASHMEMIOC, 8)
#define ASHMEM_ISPINNED		_IO(__ASHMEMIOC, 9)
#define ASHMEM_PURGE_ALL_CACHES	_IO(__ASHMEMIOC, 10)

#define MEMSZ 1024 * 1024 * 10
#define MAPSZ 1024 * 1024 * 6

int
main(int argc, const char *argv[]) {
	char  pidbuf[32];
	int   fd, cnt, done, sz;
	char *b, *map;

	b = malloc(MEMSZ);

	fd = open("/dev/ashmem", O_RDWR);
	if (fd < 0) {
		printf("Fail open ashmem\n");
		return -1;
	}

	if (0 > ioctl(fd, ASHMEM_SET_NAME, "yhc-test-mem")) {
		printf("Fail set ashmem name\n");
		return -1;
	}

	if (0 > ioctl(fd, ASHMEM_SET_SIZE, MAPSZ)) {
		printf("Fail set ashmem size\n");
		return -1;
	}

	map = (char *)mmap(NULL,
			   MAPSZ,
			   PROT_NONE,
			   MAP_PRIVATE,
			   fd,
			   0);
	if (MAP_FAILED == map) {
		printf("Map failed\n");
		return -1;
	}

	close(fd);

	/* demand half of the mmap pages */
	mprotect(map, MAPSZ / 2, PROT_WRITE);
	sz = MAPSZ / 2;
	/* access MAPSZ / 2 -> read on demand */
	while (sz--)
		map[sz] = 0xff;

	done = 0;
	cnt = 3;
	while (cnt--) {
		switch (fork()) {
		case -1:
			printf("ERR fork\n");
			exit(0);

		case 0: /* child */
			done = 1;
		}
		if (done)
			break;
	}

	while (1) {
		if (0 <= open("done", O_RDONLY))
			break;

		snprintf(pidbuf, sizeof(pidbuf), "%d-mem", getpid());
		if (0 <= (fd = open(pidbuf, O_RDONLY))) {
			close(fd);
			unlink(pidbuf);
			/* access on demand */
			memset(b, 0, MEMSZ);
		}
		sleep(1);
	}
	return EXIT_SUCCESS;
} 

You can see that PSS != RSS in ashmem memory area and it is shared among forked child process.

And there is another interesting point.
We know that static/global memory is private like memory allocated by malloc. But interestingly, VMA for this static/global memory is NOT even assigned before they are actually demanded, while VMA for dynamically allocated memory is immediately assigned.
You can easily tested this by using following code snippet.

#define MALLOCSZ 10 * 1024 * 1024
#define BSSSZ    3 * 1024 * 1024

static char sbuf[BSSSZ];

int
main(int argc, const char *argv[]) {
	char *buf;
	//sbuf[0] = 1; <--- (A)
	buf = malloc(MALLOCSZ);
	while (1) { sleep(10); }
	return 0;
}

without line (A), VMA for sbuf is NOT assigned. So, VSS size doesn't include size for sbuf.
After enabling line (A), VSS is increased by sizeof(sbuf).
On the other hand, size allocated by malloc is included at VSS even if it is NOT demanded yet.
Interesting, isn't it?

Now, it time to dive into one of deepest part in terms of memory - page.
Every process has it's own page table. And this has all about process's memory information.
Linux kernel provides various useful information regarding memory page via proc file system.
smaps is one of them.
At this step, I would like to mention about /proc/<pid>/maps, /proc/<pid>/pagemap, /proc/kpagecount and /proc/kpageflags.
To know about process's memory, you need to know about memory page used by the process.
But, lots of pages in page table doesn't have real mapping yet.
Therefore, instead of searching whole page table - this is wasting of time, we can start from smaps.
maps shows VMA (subset of data shown by smaps). And those are we are interested in.
Now, we know virtual memory address this is valid in the process.
Next step is finding corresponding pages. pagemap gives this information.
Each entry of pagemap has 64bit value. And this gives following information (from pagemap.txt in kernel document).

    * Bits 0-54  page frame number (PFN) if present
    * Bits 0-4   swap type if swapped
    * Bits 5-54  swap offset if swapped
    * Bits 55-60 page shift (page size = 1<<page shift)
    * Bit  61    reserved for future use
    * Bit  62    page swapped
    * Bit  63    page present

Most important value here is PFN. PFN is used as index at kpagecount and kpageflags.
Kernel document - pagemap.txt - says like follows (based on Kernel 3.4)

 * /proc/kpagecount.  This file contains a 64-bit count of the number of
   times each page is mapped, indexed by PFN.

 * /proc/kpageflags.  This file contains a 64-bit set of flags for each
   page, indexed by PFN.

   The flags are (from fs/proc/page.c, above kpageflags_read):

     0. LOCKED
     1. ERROR
     2. REFERENCED
     3. UPTODATE
     4. DIRTY
     5. LRU
     6. ACTIVE
     7. SLAB
     8. WRITEBACK
     9. RECLAIM
    10. BUDDY
    11. MMAP
    12. ANON
    13. SWAPCACHE
    14. SWAPBACKED
    15. COMPOUND_HEAD
    16. COMPOUND_TAIL
    16. HUGE
    18. UNEVICTABLE
    19. HWPOISON
    20. NOPAGE
    21. KSM
    22. THP

Finally, we know lots of valuable information for each pages. By combining them, we can get meaningful information - ex. USS, RSS, PSS VSS, swap etc.)
For details, you can refer kernel document (proc.txt and pagemap.txt) and source code.
IMPORTANT NOTE
See Kernel source code. Then you can easily capture that flag information in pagemap.txt is out of date.
It's up to readers to tell the difference between source and document. :-).

This is mechanism is exactly what procrank tool in Android is used.
Android 4.3 or lower has bug in libpagemap.so. So, until now, VSS is not correctly displayed by procrank.
Following code snippet is from libpagemap.so. in Android 4.3.

int pm_map_usage(pm_map_t *map, pm_memusage_t *usage_out) {
    uint64_t *pagemap;
    size_t len, i;
    uint64_t count;
    pm_memusage_t usage;
    int error;

    if (!map || !usage_out)
        return -1;

    error = pm_map_pagemap(map, &pagemap, &len);
    if (error) return error;

    pm_memusage_zero(&usage);

    for (i = 0; i < len; i++) {
        ----- line (A) -----
        if (!PM_PAGEMAP_PRESENT(pagemap[i]) ||
            PM_PAGEMAP_SWAPPED(pagemap[i]))
            continue;

        error = pm_kernel_count(map->proc->ker, PM_PAGEMAP_PFN(pagemap[i]),
                                &count);
        if (error) goto out;

        usage.vss += map->proc->ker->pagesize; // ----- line (B) -----
        usage.rss += (count >= 1) ? (map->proc->ker->pagesize) : (0);
        usage.pss += (count >= 1) ? (map->proc->ker->pagesize / count) : (0);
        usage.uss += (count == 1) ? (map->proc->ker->pagesize) : (0);
    }

    memcpy(usage_out, &usage, sizeof(usage));

    error = 0;

out:    
    free(pagemap);

    return error;
}

As you can see, page which map count == 1, is included at USS. And code for getting RSS and PSS is also easily understandable.
But, in case of VSS - line (B) - should be move to line (A) and I am sure that this bug will be fixed soon.
<--- to be continued...

Error message : Android requires .class compatibility set to 5.0. Please fix project properties.

Solution
- go to "project -> property -> Java Compiler ->"
- Set "Compiler compliance level" to 1.5. And push "Apply" button.

Now it is resolved.

Interesting thing : Once this issue is resolved by changing "Compiler compliance", even if it we restore this value back to 1.6, this error isn't reproduced. I don't have any idea about the reason.

--- Sub-shell / Current shell ---

(*1)
    echo -e "aaa\nbbb\nccc\nddd" > tmp
    value=0;
    while read line; do
        value=$(expr $value + 1)
        echo $value
    done < tmp
    rm -f tmp
    echo "at last: $value"

    result:
    1
    2
    3
    4
    at last: 4

(*2)
    value=0
    echo -e "aaa\nbbb\nccc\nddd" | while read line; do
        value=$(expr $value + 1)
        echo $value
    done
    echo "at last: $value"

    result:
    1
    2
    3
    4
    at last: 0

It is interesting, isn't it? The main reason is, (*1) is executed in current shell. But (*2) is executed in sub-shell. In more detail, there is only one process(current shell) during run at (*1). But at (*2), there are three processes; current shell, process for 'echo' and process for 'while loop'.

--- Shell function ---

(*1)
    function func_name() {
        ...
    }

(*2)
    func_name() {
        ...
    }

(*2) is more portable than (*1). Bash supports both (*1) and (*2). But, dash - /bin/sh -> dash in Ubuntu - doesn't support (*1).

--- Syntax ---
'()' grouping and '{}' grouping is delicately different in it's syntax.

(echo 1; echo 2; exit 0) # OK
{ echo a; echo b; exit 1 } # syntax error
{ echo a; echo b; exit 1; } # OK

 
--- Test ---
'!' has higher priority than '-a' and '-o'.

Assume that there are two files; 'a' and 'b'

[ -r a -a -r b ] : TRUE
[ ( -r a -a -r b ) ] : syntax error
[ ( -r a ) -a ( -r b ) ] : syntax error
[ ! -r a -o -r c ] : FALSE
[ ! -r a -o -r b ] : TRUE

 
--- Replacement ---
'~' isn't replaced in "". So,

MYHOME=~ # OK
MYHOME="~" # it doesn't work as expected

 
--- Symbolic link ---
Assume following file structure.

/data/userA/
/data/userB/
/home/a (a -> /data/userA/)
/home/b (b -> /data/userB/)

Let's see the following test.

cd /home/
cd a
pwd
    -> '/home/a' is shown.
ls ..
    -> 'userA' and 'userB' are shown. -- (*a)
cd ..
pwd
    -> '/home' is shown -- (*b)
ls
    -> 'a' and 'b' are shown. -- (*c)
cd /home/
cd a
cp a.txt ../
    -> a.txt is created at '/data/' -- (*d)

As you can see, the way handling symbolic link is different among commands. In above test, 'cd' and 'pwd' work differently from 'ls' and 'cp'. Some commands - like 'cd' and 'pwd' - handle directory path of working directory but some - like 'cp' and 'ls' - doesn't.
Therefore, try to use 'absolute path' to avoid the case like this, if symbolic link is included in the directory path.