31.7.16

Linux Processes: Part 2

Now it's time to talk about signals that are part of POSIX speicfication. This is very important, because we cannot go any further without signals theory.

2. Signals

Signals are mechanisms for communicating with and manipulating processes in Linux. The topic of signals is a large one; here we discuss some of the most important signals and techniques that are used for controlling processes.

A signal is a special message sent to a process. Signals are asynchronous; when a process receives a signal, it processes the signal immediately, without finishing the current function or even the current line of code. There are several dozen different signals, each with a different meaning. Each signal type is specified by its signal number, but in programs, you usually refer to a signal by its name. In Linux, these are defined in /usr/include/bits/signum.h. (You shouldn’t include this header file directly in your programs; instead, use "signal.h".)

When a process receives a signal, it may do one of several things, depending on the signal’s disposition. For each signal, there is a default disposition, which determines what happens to the process if the program does not specify some other behavior. For most signal types, a program may specify some other behavior—either to ignore the signal or to call a special signal-handler function to respond to the signal. If a signal handler is used, the currently executing program is paused, the signal handler is executed, and, when the signal handler returns, the program resumes.

The Linux system sends signals to processes in response to specific conditions. For instance, SIGBUS (bus error), SIGSEGV (segmentation violation), and SIGFPE (floating point exception) may be sent to a process that attempts to perform an illegal operation. The default disposition for these signals it to terminate the process and produce a core file.

A process may also send a signal to another process. One common use of this mechanism is to end another process by sending it a SIGTERM or SIGKILL signal. What’s the difference between SIGTERM and SIGKILL? The SIGTERM signal asks a process to terminate; the process may ignore the request by masking or ignoring the signal. The SIGKILL signal always kills the process immediately because the process may not mask or ignore SIGKILL.

Another common use is to send a command to a running program. Two "userdefined" signals are reserved for this purpose: SIGUSR1 and SIGUSR2. The SIGHUP signal is sometimes used for this purpose as well, commonly to wake up an idling program or cause a program to reread its configuration files.

The sigaction function can be used to set a signal disposition. The first parameter is the signal number. The next two parameters are pointers to sigaction structures; the first of these contains the desired disposition for that signal number, while the second receives the previous disposition.The most important field in the first or second sigaction structure is sa_handler. It can take one of three values:

  • SIG_DFL, which specifies the default disposition for the signal.
  • SIG_IGN, which specifies that the signal should be ignored.
  • A pointer to a signal-handler function.The function should take one parameter, the signal number, and return void.

Because signals are asynchronous, the main program may be in a very fragile state when a signal is processed and thus while a signal handler function executes. Therefore, you should avoid performing any I/O operations or calling most library and system functions from signal handlers.

A signal handler should perform the minimum work necessary to respond to the signal, and then return control to the main program (or terminate the program). In most cases, this consists simply of recording the fact that a signal occurred.The main program then checks periodically whether a signal has occurred and reacts accordingly.

It is possible for a signal handler to be interrupted by the delivery of another signal. While this may sound like a rare occurrence, if it does occur, it will be very difficult to diagnose and debug the problem. Therefore, you should be very careful about what your program does in a signal handler.

Even assigning a value to a global variable can be dangerous because the assignment may actually be carried out in two or more machine instructions, and a second signal may occur between them, leaving the variable in a corrupted state. If you use a global variable to flag a signal from a signal-handler function, it should be of the special type sig_atomic_t. Linux guarantees that assignments to variables of this type are performed in a single instruction and therefore cannot be interrupted midway. In Linux, sig_atomic_t is an ordinary int; in fact, assignments to integer types the size of int or smaller, or to pointers, are atomic. If you want to write a program that’s portable to any standard UNIX system, though, use sig_atomic_t for these global variables.

This program skeleton listed below, for instance, uses a signal-handler function to count the number of times that the program receives SIGUSR1, one of the signals reserved for application use.

#include "signal.h"
#include "stdio.h"
#include "string.h"
#include "sys/types.h"
#include "unistd.h"

static sig_atomic_t sigusr1_count = 0;

void handler (int signal_number) {
    ++sigusr1_count;
}

int main (int argc, char** argv) {
    struct sigaction sa;
    memset (&sa, 0, sizeof (sa));

    sa.sa_handler = &handler;
    sigaction (SIGUSR1, &sa, NULL);

    /* Do some cool stuff here. */
    /* ... */

    printf ("SIGUSR1 was raised [%i] times\n", sigusr1_count);
    return 0;
}

2.1 Process termination

Normally, a process terminates in one of two ways. Either the executing program calls the exit function, or the program’s main function returns. Each process has an exit code: a number that the process returns to its parent. The exit code is the argument passed to the exit function, or the value returned from main.

A process may also terminate abnormally, in response to a signal. For instance, the SIGBUS, SIGSEGV, and SIGFPE signals mentioned previously cause the process to terminate. Other signals are used to terminate a process explicitly.The SIGINT signal is sent to a process when the user attempts to end it by typing Ctrl+C in its terminal.The SIGTERM signal is sent by the kill command. The default disposition for both of these is to terminate the process. By calling the abort function, a process sends itself the SIGABRT signal, which terminates the process and produces a core file.The most powerful termination signal is SIGKILL, which ends a process immediately and cannot be blocked or handled by a program.

Any of these signals can be sent using the kill command by specifying an extra command-line flag; for instance, to end a troublesome process by sending it a SIGKILL, invoke the following, where pid is its process ID:

 % kill -KILL pid
Or more generic way just use number of signal instead of name:
 % kill -9 pid

To send a signal from a program, use the kill function.The first parameter is the target process ID. The second parameter is the signal number; use SIGTERM to simulate the default behavior of the kill command. For instance, where child pid contains the process ID of the child process, you can use the kill function to terminate a child process from the parent by calling it like this:

 kill (child_pid, SIGTERM);

Always include the "sys/types.h" and "signal.h" headers if you use the kill function.

By convention, the exit code is used to indicate whether the program executed correctly. An exit code of zero indicates correct execution, while a nonzero exit code indicates that an error occurred. In the latter case, the particular value returned may give some indication of the nature of the error. It’s a good idea to stick with this convention in your programs because other components of the GNU/Linux system assume this behavior. For instance, shells assume this convention when you connect multiple programs with the && (logical and) and || (logical or) operators. Therefore, you should explicitly return zero from your main function, unless an error occurs.

Note that even though the parameter type of the exit function is int and the main function returns an int, Linux does not preserve the full 32 bits of the return code. In fact, you should use exit codes only between zero and 127. Exit codes above 128 have a special meaning—when a process is terminated by a signal, its exit code is 128 plus the signal number.

2.2 Waiting for process termination

If you typed in and ran the fork and exec example in part 1 paragraph 1.2, where we tried to create forked process, you may have noticed that the output from the ls program often appears after the "main program" has already completed. That’s because the child process, in which ls is run, is scheduled independently of the parent process. Because Linux is a multitasking operating system, both processes appear to execute simultaneously, and you can’t predict whether the ls program will have a chance to run before or after the parent process runs.

In some situations, though, it is desirable for the parent process to wait until one or more child processes have completed. This can be done with the wait family of system calls. These functions allow you to wait for a process to finish executing, and enable the parent process to retrieve information about its child’s termination. There are three different system calls in the wait family; you can choose to get a little or a lot of information about the process that exited, and you can choose whether you care about which child process terminated.

The simplest such function is called simply wait. It blocks the calling process until one of its child processes exits (or an error occurs). It returns a status code via an integer pointer argument, from which you can extract information about how the child process exited. For instance, the WEXITSTATUS macro extracts the child process’s exit code.

You can use the WIFEXITED macro to determine from a child process’s exit status whether that process exited normally (via the exit function or returning from main) or died from an unhandled signal. In the latter case, use the WTERMSIG macro to extract from its exit status the signal number by which it died.

Here is the main function from the fork and exec example again. This time, the parent process calls wait to wait until the child process, in which the ls command executes, is finished:

#include "stdio.h"
#include "stdlib.h"
#include "sys/types.h"
#include "unistd.h"

/* Spawn a child process running a new program. PROGRAM is the name
of the program to run; the path will be searched for this program.
ARG_LIST is a NULL-terminated list of character strings to be
passed as the program’s argument list. Returns the process ID of
the spawned process. */
int spawn (char* program, char** arg_list) {
    pid_t child_pid;
    
    /* Duplicate this process. */
    child_pid = fork ();

    if (child_pid != 0) {
       /* This is the parent process. */
       return child_pid;
    } else {
       /* Now execute PROGRAM, searching for it in the path. */
       execvp (program, arg_list);
       /* The execvp function returns only if an error occurs. */
       fprintf (stderr, "an error occurred in execvp\n");
       abort ();
    }
}

int main (int argc, char** argv) {
    int child_status;

    /* The argument list to pass to the "ls" command. */
    char* arg_list[] = {
        "ls", /* argv[0], the name of the program. */
        "-l",
        "/",
        NULL /* The argument list must end with a NULL. */
    };
    
    /* Spawn a child process running the "ls" command. Ignore the returned child process ID. */
    spawn ("ls", arg_list);

    /* Wait for the child process to complete. */
    wait (&child_status);

    if (WIFEXITED (child_status)) {
        printf ("the child process exited normally, with exit code [%i]\n", WEXITSTATUS (child_status));
    } else {
        printf ("the child process exited abnormally\n");
    }

    return 0;
}

Several similar system calls are available in Linux, which are more flexible or provide more information about the exiting child process. The waitpid function can be used to wait for a specific child process to exit instead of any child process. And there are two waitpid functions. The first waitpid function returns CPU usage statistics about the exiting child process, and the second one waitpid function allows you to specify additional options about which processes to wait for.

In the third, and not the last, part of this article we will talk about process states and zombie processes.

Linux Processes: Part 1

A running instance of a program is called a PROCESS. If you have two terminals showing on your screen, then you are probably running the same terminal program twice — you have two independent terminal processes. Each terminal is probably running a shell; each running shell is another process. When you invoke a command from a shell, the corresponding program is executed in a new process; the shell process resumes when that process completes.

Advanced programmers often use multiple cooperating processes in a single application to enable the application to do more than one thing at once, to increase application robustness, and to make use of already-existing programs.

Most of the process manipulation functions described in this post are similar to those on other UNIX systems. Most are declared in the header file "unistd.h"; check the man page for each function to be sure.

1. Looking at Processes

Even as you sit down at your computer, there are processes running. Every executing program uses one or more processes. Let’s start by taking a look at the processes already on your computer.

1.1 What are the process IDs?

Each process in a Linux system is identified by its unique process ID, sometimes referred to as pid. Process IDs are unsigned 16-bit integer numbers that are assigned sequentially by Linux as new processes are created.

Every process also has a parent process (except the special init process, described in section “Zombie Processes”). Thus, you can think of the processes on a Linux system as arranged in a tree, with the init process at its root. The parent process ID, or ppid, is simply the process identity of the process’s parent.

When referring to process IDs in a C or C++ program, always use the pid_t typedef, which is defined in "sys/types.h". A program can obtain the process ID of the process it’s running in with the getpid() system call, and it can obtain the process ID of its parent process with the getppid() system call. For instance, the program in Listing 1 prints its process ID and its parent’s process ID.

#include "stdio.h"
#include "unistd.h"

int main(int argc, char** argv) {
    printf( "The process id is: '%i'\n", getpid() );
    printf( "The parent process id is: '%i'\n", getppid() );
    return 0;
}

Observe that if you invoke this program several times, a different process ID is reported because each invocation is in a new process. However, if you invoke it every time from the same shell, the parent process ID (that is, the process ID of the shell process) is the same.

1.2 Viewing active processes

The ps command displays the processes that are running on your system. The GNU/Linux version of ps has lots of options because it tries to be compatible with versions of ps on several other UNIX variants. These options control which processes are listed and what information about each is shown.

By default, invoking ps displays the processes controlled by the terminal or terminal window in which ps is invoked. For example:

  PID TTY       TIME CMD
21693 pts/8 00:00:00 bash
21694 pts/8 00:00:00 ps

This invocation of ps shows two processes.The first, bash, is the shell running on this terminal. The second is the running instance of the ps program itself. The first column, labeled PID, displays the process ID of each one.

For a more detailed look at what’s running on your GNU/Linux system, invoke this:

 % ps -e -o pid,ppid,command 

The -e option instructs ps to display all processes running on the system.The -o pid,ppid,command option tells ps what information to show about each process — in this case, the process ID, the parent process ID, and the command running in this process.

ps Output Formats
With the -o option to the ps command, you specify the information about processes that you want in the output as a comma-separated list. For example, ps -o pid,user,start_time,command displays the process ID, the name of the user owning the process, the wall clock time at which the process started, and the command running in the process. See the man page for ps for the full list of field codes. You can use the -f (full listing), -l (long listing), or -j (jobs listing) options instead to get three different preset listing formats.

Here are the first few lines and last few lines of output from this command on my system.You may see different output, depending on what’s running on your system.

   PID PPID  COMMAND
     1    0  init [5]
 ....................
 21724 21693 terminator
 21727 21725 bash
 21728 21727 ps -e -o pid,ppid,command

Note that the parent process ID of the ps command, 21727, is the process ID of bash, the shell from which I invoked ps.The parent process ID of bash is in turn 21725, the process ID of the terminator (variation on terminal wrapper) program in which the shell is running.

Anyway we are talking too much on process creation and viewing it's current state, but what about...

1.3 Creating process using fork and exec

UNIX family of operating system provides set of functions, fork(), that makes a child process that is an exact copy of its parent process. Linux provides another set of functions, the exec() family, that causes a particular process to cease being an instance of one program and to instead become an instance of another program. To spawn a new process, you first use fork to make a copy of the current process. Then you use exec to transform one of these processes into an instance of the program you want to spawn.

When a program calls fork, a duplicate process, called the child process, is created. The parent process continues executing the program from the point that fork was called. The child process, too, executes the same program from the same place.

So how do the two processes differ? First, the child process is a new process and therefore has a new process ID, distinct from its parent’s process ID. One way for a program to distinguish whether it’s in the parent process or the child process is to call getpid. However, the fork function provides different return values to the parent and child processes—one process "goes in" to the fork call, and two processes "come out" with different return values. The return value in the parent process is the process ID of the child. The return value in the child process is zero. Because no process ever has a process ID of zero, this makes it easy for the program whether it is now running as the parent or the child process.

The next listing is an example of using fork to duplicate a program’s process. Note that the first block of the if statement is executed only in the parent process, while the else clause is executed in the child process.

#include "stdio.h"
#include "sys/types.h"
#include "unistd.h"

int main (int argc, char** argv) {
    pid_t child_pid;

    printf ("The main program process ID is %d\n", getpid ());
    child_pid = fork ();

    if (child_pid != 0) {
        printf ("This is the parent process, with id: [%i]\n", (int) getpid ());
        printf ("The child’s process ID is: [%i]\n", child_pid);
    } else {
        printf ("This is the child process, with id: [%i]\n", getpid ());
    }

    return 0;
}

The exec functions replace the program running in a process with another program. When a program calls an exec function, that process immediately ceases executing that program and begins executing a new program from the beginning, assuming that the exec call doesn’t encounter an error.

Within the exec family, there are functions that vary slightly in their capabilities and how they are called.

  • Functions that contain the letter 'p' in their names (execvp and execlp) accept a program name and search for a program by that name in the current execution path; functions that don’t contain the 'p' must be given the full path of the program to be executed.
  • Functions that contain the letter 'v' in their names (execv, execvp, and execve) accept the argument list for the new program as a NULL-terminated array of pointers to strings.
  • Functions that contain the letter 'l' (execl, execlp, and execle) accept the argument list using the C language’s varargs mechanism.
  • Functions that contain the letter 'e' in their names (execve and execle) accept an additional argument, an array of environment variables.The argument should be a NULL-terminated array of pointers to character strings. Each character string should be of the form "VARIABLE=value".

Because exec replaces the calling program with another one, it never returns unless an error occurs.

The argument list passed to the program is analogous to the command-line arguments that you specify to a program when you run it from the shell. They are available through the argc and argv parameters to main. Remember, when a program is invoked from the shell, the shell sets the first element of the argument list (argv[0]) to the name of the program, the second element of the argument list (argv[1]) to the first command-line argument, and so on. When you use an exec function in your programs, you too should pass the name of the function as the first element of the argument. list.

A common pattern to run a subprogram within a program is first to fork the process and then exec the subprogram.This allows the calling program to continue execution in the parent process while the calling program is replaced by the subprogram in the child process.

The next example shows invokes the ls command directly, passing it the command-line arguments -l and / rather than invoking it through a shell.

#include "stdio.h"
#include "stdlib.h"
#include "sys/types.h"
#include "unistd.h"

/* Spawn a child process running a new program. PROGRAM is the name
of the program to run; the path will be searched for this program.
ARG_LIST is a NULL-terminated list of character strings to be
passed as the program’s argument list. Returns the process ID of
the spawned process. */
int spawn (char* program, char** arg_list) {
    pid_t child_pid;
    
    /* Duplicate this process. */
    child_pid = fork ();

    if (child_pid != 0) {
       /* This is the parent process. */
       return child_pid;
    } else {
       /* Now execute PROGRAM, searching for it in the path. */
       execvp (program, arg_list);
       /* The execvp function returns only if an error occurs. */
       fprintf (stderr, "an error occurred in execvp\n");
       abort ();
    }
}

int main (int argc, char** argv) {
    /* The argument list to pass to the "ls" command. */
    char* arg_list[] = {
        "ls", /* argv[0], the name of the program. */
        "-l",
        "/",
        NULL /* The argument list must end with a NULL. */
    };
    
    /* Spawn a child process running the "ls" command. Ignore the returned child process ID. */
    spawn ("ls", arg_list);
    printf ("done with main program\n");
    return 0;
}

1.4 Killing process

You can kill a running process with the kill command. Simply specify on the command line the process ID of the process to be killed.

The kill command works by sending the process a SIGTERM, or termination, signal. This causes the process to terminate, unless the executing program explicitly handles or masks the SIGTERM signal. Signals are described in part two of this article.

28.2.16

The way to C#: microDI.NET

TL;DR: there is another DI library for C#, it's here - https://bitbucket.org/pavel-xrayz13/microdi.net - use it as you want.

It was interesting for me to start small project in another programming language just for fun. So here it is: microDI.NET.

It's focused on small applications and main idea is to be tiny as much as could it be. There are no plans to confront with huge ones like NInject or Castle, because they are already here with huge auditory and with a lots of cool features. Yes, you can use in big projects, but the library might not satisfy your needs.

So what are my primary two goals:
  1. Make it so simple as I could
  2. Easy to extend and modify

Simplification meaning is tiny in size and the code is easy to understand. "Easy to extend and modify" speaks for itself, but another thing what I'd like to raise is documentation. The last goal is also about clean documentation, because I've seen a lot of frameworks and libraries that have really crude docs. Documentation of microDI is still poor(because it's available only in README.md file in repository) for now, but I'm working on it.
Also the whole library is licensed under MIT License terms, so you are free to use it as you want.
Plans for the future:
  • Full documentation
  • Support for auto-disposing IDisposable objects
  • More unit tests
  • Performance improvements

15.11.14

Just thinking: modern GAPI

Disclaimer: this is another useless post without much of technical information.

I wanted to place this text in previous post, but that was hard to do logical division, so I devided into two posts. This post will store my thoughts about graphics APIs and their evolutions/revolutions. In discent times we programmed GAPI ourself, that gave as direct-to-metal access: sounded good, but it was impractical and there were much problem in case of porting to another hardware. So Microsoft made Direct3D for such graphics abstraction, then SGI made IrisGL which was opened later and transformed to OpenGL. Both of them are gold standard for graphics. They do a lot to make it easier to program graphics, and we can see the richest graphics in games today(like CrySis by CryTek or Battlefield by DICE). Anyway abstraction is made for simplification, but graphics made richer and now we have another problem: huge CPU usage in high numbers of draw calls.

That's why there is Mantle by AMD and Metal by Apple(only for iOS for now), also is Direct3D12 on the way. They are made for dropping out this overhead of draw calls by moving more work to programmers. It is really funny to see that the idea of "low-level" Direct3D12 is back to Direct3D1: we are making command buffer, then we are making commands and then driver consumes this buffer. The same we might say about Mantle and Metal - the are too "low-level" with somewhat "same" functionality.

Alex St. John has written a good post with comparison of Direct3D and Metal, and thoughts about OpenGL. Also, I'd like to talk about OpenGL too... It's stuck. Literally. I know Khronos Group tries to push OpenGL to the top of graphics industry, they are making a good job(but OpenGL is still lacking some cool features, like context-independent concurrency), but the main problems are IHVs. Every IHV is making it's own implementation, because OpenGL is only the specification and it doesn't have run-time component like Direct3D has. That's really bad that no one in Khronos who could bang the table and say "that must be as I say!" to stop this noncense of varying OpenGL implementations. I don't want to talk too much about this, because thare a lot of negative posts that already have the same opinion about OpenGL. Anyway, I'll still wait for OpenGL NG, I believe that Khronos Group will make it.

That's enough of "void" talking, let's consider my thoughts about modern graphics API.
First of all, API must provide one view of resource as raw memory. This means that we will work with it like in our programs via malloc/realloc/free, just like in CUDA. Second thought is continuation of the first, we must control allocated chunk of memory by ourselves, making needed memory barriers or doing sequental consistency read/write operations and etc. Third is that API must provide controlling interface of GPU scheduler. That would bring us continous kernel execution like in CUDA, where we can write kernel that has ability to launch other kernel functions. The third means that we will not be CPU-bounded. Ideally it would be the greatest thing to do programming of GPU like CPU. The best reference to this approach is CUDA. Also it's good to see something

That's all what I want to say. This is the last useless post, next time I'll write about our engine.

30.6.14

It was so quiet here...

...because there were not any reasons to write another useless/pointless post. Now I have a lot of information to say here, so let's start.

Engine update:
NanoEngine will be open-sourced forever. Yes, that's it. It will be under BSD license. For now, the engine is in Bitbucket's repository and it's not complete. Foundation is the most stable and complete module of the engine, while others are not, or are in draft quality on my PC. The complete list of modules will be:
    - Foundation: where all low-level(in terms of engine) routines, like compiler definitions, containers, memory allocators, threading primitives, multitasking system, etc.

    - Rendering, Audio, Input: these are foundation-like low-level implementations, which wraps selected APIs in common form. They are just back-ends for high-level implementations.

    - Framework: one of clue elements of the engine, it glues all low-level modules and boots them up for further use with higher level modules. There is also central dispatching engine, which automatically calculates count of jobs to be used with

    - RML: is a centralized resource management layer, is the place, where all resources are processed asynchronously.

    - Graphics: high-level rendering implementation, based on Rendering module with more specific implementations of features needed by application.

    - Sound: high-level audio implementation, based on Audio module, which features event-based audio playing, custom sound effects, which are applied specifically to sounds or places where sounds are playing.

    - Physics: is a wrapper around ODE calls.

    - Game: the highest-level of the engine, which implements game mechanics and scripting basics.

    - Launcher: there is nothing to say much, it's just a launcher which starts up Framework module. All platform-specific actions for window are connected here with other parts of the engine, like Rendering and Input.

Current progress (26.06.2014):
    - Foundation: ~90% - (In repository)
    - Rendering: ~70% (In repository)
    - Audio: ~30% (In repository)
    - Input: ~10% (Not in reposity)
    - Framework: ~20% (In repository)
    - RML: ~50% (Not in repository)
    - Graphics: ~30% (Not in repository)
    - Sound: N/A
    - Physics: ~10% (Not in repository)
    - Game: ~10% (Not in repository)
    - Launcher: ~70% (In repository)


Engine code:
There are some things which are always colliding: code complexity and clarity of code. I am using a lot of templates in the engine. Yes, templates are one of cool features of the language, but it's also vital to use them where they are needed.
Let's consider this example from the code:
/// Intrusive smart pointer declaration
template< typename Type, typename Policy > class IntrusivePtr;
/// Policy declaration
template< typename Type, typename Tag > class SmartPtrPolicy;
/// Policy tags
struct SmartPointerRelaxedTag {};
struct SmartPointerThreadSafeTag {};

/// Use case example
IntrusivePtr< MyClass, SmartPtrPolicy< MyClass, SmartPointerRelaxedTag > > myClassPtr;
As you see, there is big problem with templates(in case of text complexity). But they are used for synchronization specialization, for automatic code path selection in compile-time for threading purposes. One might say, it's better to change this from full type-templated specialization to value-templated, to make something like this:
template< typename Type, Threading::MemoryOrder accessOrder, Threading::MemoryOrder writeOrder > class IntrusivePtr;
// Use case example
IntrusivePtr< MyClass, Threading::MemoryOrderEnum::Acquire, Threading::MemoryOrderEnum::Release > myClassPtr;
As you see, it is as bad as...I don't know...as the worst thing that you may imagine: too many chars to write, too big length of string, so it would be harder to read and understand code because of "extreme navigation". Another pro might be, automatic exclusion of dead code in compile-time, but not every compiler will do it, because you need to specify it yourself. In Visual C++ you can specify /O2, but this would work for release version, and not debug, where you want to explore the whole assembled code without any optimizations. So, that's why I'm using typed-templated version with tag selector: the old good method, which is not abandoned nowadays.

It was something like introduction to engine's specialization. I made such example with templated smart pointer to show that engine will make use of such pointers for automatic garbage collection. There various types of them: IntrusivePtr with intrusive reference counting in class and deletion is also needs to be specified inside class, SharedPtr with self-allocated reference couting. Both of them use SmartPtrPolicy for threading model selection.
As it was mentioned in Engine Update section, the engine aimed at agressive multithreading, so we need containers for inter-thread communications. There are various types of lists, queues and rind buffer implementations, which are Lockfree SPSC(Single-producer/single-consumer), Lockfree MPSC(Multi-producer/single-consumer), Lockfree MPMC(Multi-producer/multi-consumers), and Blocking ones.

That's all for code section now, because I don't want to share more details before uploading new version into Git. Also, if you want to see code, I'll give an access to our git storage, just email me: techzdev@4dvisionzgamesz.com(remove all "z")

21.7.12

Some additional notes to the previous post

Multithreading in OpenGL is a hard thing, event with Fence extension, this may "eat" all CPU processing time, because of sync overhead - this is multipy contexts version. But we can have only one context attached to seaparate thread and command buffering for execution by that thread. One rendering command for command buffer looks like this:
struct rendering_command
{
    rendering_operation op;
    unsigned int gl_id;
    unsigned char* data;
}
Let's see, what do we have here:
1)rendering_operation is a single command for OpenGL rendering system. Just to specify event.
2)gl_id is an OpenGL object identifier.
3)data is a pointer to the data storage need by this rendering event.

This is just very simple rendering command definition, you can make it more complex by adding storage for some additional data, used by renderer. You can add unsigned int param1, param2, param3 to structure to store parameters like texture size, when you want to schedule data to upload to texture or modify, where param1 is a width, param2 - height, param3 - depth.
For command buffer storage you can use queue, but it would be better to make it thread-safe via critical sections or mutexes to prevent "false threading" situation, or just use concurrent_queue from Intel Threading Building blocks[1].

Conclusion is just simple: there is no standard for such multithreading situation with OpenGL. You can use method proposed by NVIDIA in "Optimized OpenGL texture transfers"[2], or use method proposed here. Small chunks of textures(1x1 - 64x64) works great with this method, for large it's better to use NVIDIA's.

References:
[1]: http://threadingbuildingblocks.org/
[2]: http://nvidia.fullviewmedia.com/gtc2012/0515-J2-S0356.html

Optimized texture transfers with OpenGL


Updated: see some additional notes to this post