Jump to content
Sign in to follow this  
yyrkoon

Linux stack size.

Recommended Posts

Today I was faced with a problem that cause a bit of code to stop working using dynamic memory. So I picked up a book I had laying around and started reading the memory allocation section - Since the internet was down for us . . . then started reading the section on alloca(), and realized that using alloca() can have some decent benefits when compared to malloc() under some conditions . . . memory allocation speed being one, which since I'll be writting this code to run on an embedded hardware platform, and I'm going to be doing a lot of allocating of memory in real time . . .

 

Which got me thinking again about ulimit - Setting a system wide stack size larger than ( 8MB for me ) that which is set by default. How does this effect the applications I'm currently running ? Does it make sense to set this value in an application that perhaps I really do not wish to use dynamic memory for ? What are the consequences ?

 

I was wondering if anyone here has had experience with a situation like this, and if so would love to hear what that / those person / people would like to share

Share this post


Link to post
Share on other sites

Any application has a separate stack from any other application. So if one application was running with a large stack, that just means it's using more memory. To other applications it does not matter whether this memory is stack (auto memory) or heap (free space).

I barely ever find the need to use alloca() ever, in particular when using C instead of C++ (yes, those are not 100% compatible!)

C has this cool thing that you can use a variable as the size for your arrays. Since variables can only be declared at the start of a block, you mostly use function parameters for this.

int foo (unsigned int bar)
{
  int baz[bar] = {0}; // only legal in C, not it C++
  ...
}

Share this post


Link to post
Share on other sites

 

Any application has a separate stack from any other application. So if one application was running with a large stack, that just means it's using more memory. To other applications it does not matter whether this memory is stack (auto memory) or heap (free space).

I barely ever find the need to use alloca() ever, in particular when using C instead of C++ (yes, those are not 100% compatible!)

C has this cool thing that you can use a variable as the size for your arrays. Since variables can only be declared at the start of a block, you mostly use function parameters for this.

int foo (unsigned int bar)
{
  int baz[bar] = {0}; // only legal in C, not it C++
  ...
}

I find that kind of funny - That sudden realization that using alloca() is really no more than that which one was trying to escape when they start using malloc(),  Anyway, that realization came to me about two seconds after implementing it into my code heh.

 

The reality is though, in total I'm barely using 1,8xx Bytes heap as reported by valgrind. So in effect, I could probably just use all local variables, and not have to worry too much.  Maybe I'm missing something here, which wouldn't be much of a surprise. If I did want to share the code with "the world" is where it would probably be a problem. Aside from that, stack allocation does not work the way it once used to on Linux. In my own case, using Debian, my ulimit is technically unlimited per process - By default. 8MB is just how much the system gives each user initially( per process ), with the ability to grow as needed. Unless configured otherwise. Still as valgrind reported, it seems largely moot anyway.

 

So I could possibly refactor out the use of alloca(), and instead use a local char *. Since in my code all I am really doing is creating a temp struct object, with two members( one being a char *buffer ), and then creating a union boxed struct + char *buffer set, and passing that back to the caller by reference.  So maybe the following will clear some  things up ?

struct shm_data {
    char *data;
    int file_lock : 8;
};

typedef union{
    struct shm_data sdata;
    char *mmap_addr;
}shm_t;

...

shm_t *InitializeShm(const long size, const int fd){
    struct shm_data *sms = alloca(size);
    if(sms == NULL)
        HandleError("Unable to create data structure shm_data *sms.");
    
    sms->file_lock = 0;

    shm_t *shm = (shm_t *)sms;
    if(shm == NULL)
        HandleError("Unable to create union *shm");
    
    shm->mmap_addr = mmap(NULL, size, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
    if (shm->mmap_addr == MAP_FAILED)
        HandleError(strerror(errno));
    
    close(fd);

    return shm;
}

Rereading my code here, I *think* I can probably do away with the alloca() call and that whole struct line plus error checking. As now that I think on it - It seems redundant / not necessary.

Share this post


Link to post
Share on other sites

Note that such code is really dangerous! Since you alloca() (stack space) the memory inside your InitializeShm() function, the compiler is free to repurpose that memory right at the point where the return is.

There is no guarantee at all that the space you're getting a pointer to (in the caller to InitializeShm()) is in fact a legal part of your stack.

As such, your compiler is free to store any temporary values and stuff like that in the same space as InitializeShm() has given you a pointer to.

Share this post


Link to post
Share on other sites

C has this cool thing that you can use a variable as the size for your arrays. Since variables can only be declared at the start of a block, you mostly use function parameters for this.

int foo (unsigned int bar)
{
  int baz[bar] = {0}; // only legal in C, not it C++
  ...
}

 

FYI, that was introduced in C99. It's not part of the K&R, ANSI C or C90 standards.

Share this post


Link to post
Share on other sites

@@roadrunner84

 

Note that such code is really dangerous! Since you alloca() (stack space) the memory inside your InitializeShm() function, the compiler is free to repurpose that memory right at the point where the return is.

There is no guarantee at all that the space you're getting a pointer to (in the caller to InitializeShm()) is in fact a legal part of your stack.

As such, your compiler is free to store any temporary values and stuff like that in the same space as InitializeShm() has given you a pointer to.

 

Oh, most definitely ! There, I'm using alloca() *only* to initialize the struct I then later cast into the union boxed object. Well 3 lines later heh( or something like that ). I tried variations of directly allocating the struct within the union, but kept getting segmentation faults . . . This is the only way I've found so far to initialize the union->struct.

 

What this tells me though, is that I'm probably going about doing this the wrong way . . . How so, not sure.Yet.

 

EDIT:

 

Roadrunner. Honestly, this had been on my mind a couple days ago, and I was worried about it at that time. But got busy with other things, and since put it out of my mind. Since the code seems to run so well, and fast. However, I *really really* do not want to smash the stack heh !

 

So maybe I'll refactor the code to pass the object back by value, instead of reference. If that's what it takes. e.g. removing all alloc()'s period.

Share this post


Link to post
Share on other sites

FYI, that was introduced in C99. It's not part of the K&R, ANSI C or C90 standards.

Not much of a problem here, since the version of gcc I'm using is C11 compliant, and I believe is what it's set to by default. I mean, I'd like to share my code sure, but I'm not going to use an older standard . . . So if someone wants to re-implement my code into C89, or whatever. Then that's on them.

 

EDIT: But to be perfectly honest, I've read a bit on each standard. Here and there. And I could not tell you a key feature from either one. I'm just kind of plodding along . . . reading, seeing something that would work for my current situation. Then experiment. One of the cool things of being a hobbyist, versus a professional ;)

Share this post


Link to post
Share on other sites

So, RoadRunner. How would I check what is actually happening behind the scenes ? I've had the code running all night. Which is to say two separate processes ( IPC server / client - If you've been following my socketCAN story . . ). Both use the same initialize function.

 

The server spits data out into the file about 3-4 times a second. Where the client grabs it as soon as it's available, and sends the data out over a websocket.

 

With that said, both programs seem to be working flawlessly, and no noticible side effects . . . atop follows:

ATOP - debian-can                 2015/07/24  15:02:37                 ------                   10s elapsed
PRC |  sys    2.68s  |  user   0.00s  |  #proc     74  |  #tslpu     0  |  #zombie    0  |  #exit      0  |
CPU |  sys      16%  |  user      0%  |  irq       2%  |  idle     82%  |  wait      1%  |  curscal   ?%  |
CPL |  avg1    0.00  |  avg5    0.05  |  avg15   0.11  |  csw     9802  |  intr    2783  |  numcpu     1  |
MEM |  tot     3.5G  |  free    3.3G  |  cache 117.4M  |  dirty   0.0M  |  buff   15.6M  |  slab    9.5M  |
SWP |  tot     4.1G  |  free    4.1G  |                |                |  vmcom  70.4M  |  vmlim   5.8G  |
DSK |           sda  |  busy      1%  |  read       0  |  write      3  |  MBw/s   0.00  |  avio 18.7 ms  |
NET |  transport     |  tcpi      82  |  tcpo      82  |  udpi       7  |  udpo       1  |  tcpao      0  |
NET |  network       |  ipi       88  |  ipo       83  |  ipfrw      0  |  deliv     88  |  icmpo      0  |
NET |  eth0      0%  |  pcki     107  |  pcko      83  |  si    5 Kbps  |  so    9 Kbps  |  erro       0  |
NET |  vcan0   ----  |  pcki    2474  |  pcko    2474  |  si   14 Kbps  |  so   14 Kbps  |  erro       0  |

  PID RUID     EUID      THR  SYSCPU  USRCPU  VGROW  RGROW  RDDSK  WRDSK ST EXC S CPUNR  CPU CMD        1/1
30719 william  william     1   2.34s   0.00s     0K     0K     0K     0K --   - R     0  60% canplayer
 4675 william  william     1   0.17s   0.00s     0K     0K     0K     0K --   - S     0   4% test
 4624 william  william     1   0.11s   0.00s     0K     0K     0K     0K --   - S     0   3% run
    3 root     root        1   0.03s   0.00s     0K     0K     0K     0K --   - S     0   1% ksoftirqd/0
 4364 root     root        1   0.01s   0.00s     0K     0K     0K     0K --   - R     0   0% atop
28152 william  william     1   0.01s   0.00s     0K     0K     0K     0K --   - S     0   0% sshd
    1 root     root        1   0.01s   0.00s     0K     0K     0K     0K --   - S     0   0% init
  145 root     root        1   0.00s   0.00s     0K     0K     0K     4K --   - S     0   0% jbd2/sda1-8

I also ran valgrind . . . but we're not talking heap here . . .

 

EDIT:

Now, I've replaced alloca() with malloc(), and the code seems to run fine . . .weird. Something in my *New(), and / or *Delete() functions must have been amiss. But I checked them many times . . .

 

As an additional note. I'm not using free() on these objects( malloc() ). Since everything I've read says that once the application finishes, the OS will take care of this automatically. Also since this object is in use, and in memory for the entire life of the application. I figure no harm, no foul . . .

Share this post


Link to post
Share on other sites

doing malloc() and omitting free() is okay if it's indeed for the entire run of the application, though free()ing would be cleaner, it's in no sense required then.

*gazes at atop* no clue.

alloca() might just work fine, as I said, but you might just screw up everything....

you have a union with a struct in it, are you using a union to share memory between mutually exclusive variables (what unions are meant for) or to make some casting method (don't!!)?

In the latter case, please just use a cast instead.

Share this post


Link to post
Share on other sites

doing malloc() and omitting free() is okay if it's indeed for the entire run of the application, though free()ing would be cleaner, it's in no sense required then.

*gazes at atop* no clue.

alloca() might just work fine, as I said, but you might just screw up everything....

you have a union with a struct in it, are you using a union to share memory between mutually exclusive variables (what unions are meant for) or to make some casting method (don't!!)?

In the latter case, please just use a cast instead.

I'm using the union to map the buffer to a multi variable structure. In order to get at the actual data in human readable form.  The buffer, which can be varied in length, is formed 8 bytes at a time - which is all that socketCAN read() returns per frame. Casting ? You know, I hoestly do not know. I can see how this could be viewed as such, but see no other way other than iterating through the whole buffer, and . . .  yeah I'd view that as overly complex / clunky.

 

I do know that what I'm doing is not exactly portable. But not sure how to this in a portable way. Maybe ntol() and such but hmm, not sure.

 

EDIT:

 

Sorry, the above is not really related to this topic here. Too much code on the brain. Anyhow I'm basically doing the same as mentioned above. But the structure I'm mapping to is a representation of a mmap()'d file with the last byte being used as a file locking mechanism. #define LOCKED 1 \ #define UNLOCKED 0, etc. Simply, because I could not wrap my brain around semaphores in a reasonable amount of time. Plus they seem far to complex for my needs.

 

So in this case the union is mapping a struct with two members. char *buffer, and int file_locked : 8, and it seems to work fine. Works perfectly for my own needs anyway.

Share this post


Link to post
Share on other sites

By the way. Just to be clear. I'm not sure what I'm doing as mentioned in my last post is the best way to go about my end goal. I do have a good amount of experience in C, but I'm a self taught hobbyist. Obviously I can not, and do not know everything. So if you, or anyone for that matter sees my code as bad, garbage, or whatever. I'm all ears. I do not mind constructive criticism one bit.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×