Determine Free Ram In Python

January 30, 2024 Post a Comment

I would like my python script to use all the free RAM available but no more (for efficiency reasons). I can control this by reading in only a limited amount of data but I need to

Solution 1:

On Linux systems I use this from time to time:

defmemory():
    """
    Get node total memory and memory usage
    """withopen('/proc/meminfo', 'r') as mem:
        ret = {}
        tmp = 0for i in mem:
            sline = i.split()
            ifstr(sline[0]) == 'MemTotal:':
                ret['total'] = int(sline[1])
            elifstr(sline[0]) in ('MemFree:', 'Buffers:', 'Cached:'):
                tmp += int(sline[1])
        ret['free'] = tmp
        ret['used'] = int(ret['total']) - int(ret['free'])
    return ret

You can run this when your script starts up. RAM is usually used and freed pretty frequently on a busy system, so you should take that into account before deciding how much RAM to use. Also, most linux systems have a swappiness value of 60. When using up memory, pages that are least frequently used will be swapped out. You may find yourself using SWAP instead of RAM.

Hope this helps.

Solution 2:

You could just read out /proc/meminfo. Be aware that the "free memory" is usually quite low, as the OS heavily uses free, unused memory for caching.

Also, it's best if you don't try to outsmart your OS's memory management. That usually just ends in tears (or slower programs). Better just take the RAM you need. If you want to use as much as you can on a machine with a previously unknown amount of memory, I'd probably check how much RAM is installed (MemTotal in /proc/meminfo), leave a certain amount for the OS and as safety margin (say 1 GB) and use the rest.

Solution 3:

Another option is the built-in Python package psutil:

stats = psutil.virtual_memory()  # returns a named tuple
available = getattr(stats, 'available')
print(available)

According to the documentation, the available field "is calculated by summing different memory values depending on the platform and it is supposed to be used to monitor actual memory usage in a cross platform fashion."

Note the return value will be in bytes

Solution 4:

Since the original poster wrote the code should run on a variety of Linux system, I am posting here an Object oriented solution for Linux Memory query. psutil is a great library, but if you can't install it for some reason, you can simply use the following solution:

Example usage:

>>>f = FreeMemLinux()>>>print f.total, f.used,  f.user_free
8029212 3765960 4464816
>>>>>>f_mb = FreeMemLinux(unit='MB')>>>print f_mb.total, f_mb.used,  f_mb.user_free
7841.02734375 3677.6953125 4360.171875
>>>>>>f_percent = FreeMemLinux(unit='%')>>>print f_percent.total, f_percent.used, f_percent.user_free
100.0 46.9032328453 55.60715049

Code:

classFreeMemLinux(object):
    """
    Non-cross platform way to get free memory on Linux. Note that this code
    uses the `with ... as`, which is conditionally Python 2.5 compatible!
    If for some reason you still have Python 2.5 on your system add in the
head of your code, before all imports:
    from __future__ import with_statement
    """def__init__(self, unit='kB'):

        withopen('/proc/meminfo', 'r') as mem:
            lines = mem.readlines()

        self._tot = int(lines[0].split()[1])
        self._free = int(lines[1].split()[1])
        self._buff = int(lines[2].split()[1])
        self._cached = int(lines[3].split()[1])
        self._shared = int(lines[20].split()[1])
        self._swapt = int(lines[14].split()[1])
        self._swapf = int(lines[15].split()[1])
        self._swapu = self._swapt - self._swapf

        self.unit = unit
        self._convert = self._factor()

    def_factor(self):
        """determine the convertion factor"""if self.unit == 'kB':
            return1if self.unit == 'k':
            return1024.0if self.unit == 'MB':
            return1/1024.0if self.unit == 'GB':
            return1/1024.0/1024.0if self.unit == '%':
            return1.0/self._tot
        else:
            raise Exception("Unit not understood")

    @propertydeftotal(self):
        return self._convert * self._tot

    @propertydefused(self):
        return self._convert * (self._tot - self._free)

    @propertydefused_real(self):
        """memory used which is not cache or buffers"""return self._convert * (self._tot - self._free -
                                self._buff - self._cached)

    @propertydefshared(self):
        return self._convert * (self._tot - self._free)

    @propertydefbuffers(self):
        return self._convert * (self._buff)

    @propertydefcached(self):
        return self._convert * self._cached

    @propertydefuser_free(self):
        """This is the free memory available for the user"""return self._convert *(self._free + self._buff + self._cached)

    @propertydefswap(self):
        return self._convert * self._swapt

    @propertydefswap_free(self):
        return self._convert * self._swapf

    @propertydefswap_used(self):
        return self._convert * self._swapu

Solution 5:

I suppose you could use free (http://www.cyberciti.biz/faq/linux-check-memory-usage/), ps or maybe the MemoryMonitor class from the SO-thread posted at the very bottom of my answer to do this. Just cut some slack and leave some small amount of ram unused for other processes if they should need it urgently yo avoid disk writes on their behalf.

You will need to parse the output from free or ps if you use that, but that shouldn't be hard. Remember that you need to analyze available ram real time, so you can adjust if another process gets memory hungry for some reason.

also see this thread: How to get current CPU and RAM usage in Python?

Python Freelancers