Skip to main content
Mod Moved Comments To Chat
edited body
Source Link
terdon
  • 252.7k
  • 69
  • 481
  • 719

Or, it's clever on systems where these temporary files are not themselves held in RAM, which they typically are these days (/tmp/ is typically a tmpfs, which is a RAM-only file system). So, writing these temporary files eats exactly the RAM you're trying to safesave, and you're running out of RAM: your file has 160 million lines, and a quick google suggests it's 11GB of uncompressed data.

Or, it's clever on systems where these temporary files are not themselves held in RAM, which they typically are these days (/tmp/ is typically a tmpfs, which is a RAM-only file system). So, writing these temporary files eats exactly the RAM you're trying to safe, and you're running out of RAM: your file has 160 million lines, and a quick google suggests it's 11GB of uncompressed data.

Or, it's clever on systems where these temporary files are not themselves held in RAM, which they typically are these days (/tmp/ is typically a tmpfs, which is a RAM-only file system). So, writing these temporary files eats exactly the RAM you're trying to save, and you're running out of RAM: your file has 160 million lines, and a quick google suggests it's 11GB of uncompressed data.

fix double negative
Source Link
Zombo
  • 1
  • 7
  • 47
  • 65

is not unlikelylikely to be as fast, and bonus, you get an actual database file place.sqlite. You can play around with that much more flexibly – for example, create a table where you extract coordinates, and convert the times to numerical timestamps, and then be much faster and more flexible by what you analyze.

is not unlikely to be as fast, and bonus, you get an actual database file place.sqlite. You can play around with that much more flexibly – for example, create a table where you extract coordinates, and convert the times to numerical timestamps, and then be much faster and more flexible by what you analyze.

is likely to be as fast, and bonus, you get an actual database file place.sqlite. You can play around with that much more flexibly – for example, create a table where you extract coordinates, and convert the times to numerical timestamps, and then be much faster and more flexible by what you analyze.

added 81 characters in body
Source Link
Marcus Müller
  • 52.9k
  • 4
  • 80
  • 123

TL;DR: out-of-memory-killer or running out of disk space for temporary files kills sort. Because sort tool, trying to limit its RAM usage, assumes temporary files are not using RAM. On most modern Linuxes, they are. Recommendation: Use a different tool.

You can "help" sort around that by changing the temporary directory it uses. Gonna let you lookYou're already doing that up yourself, because I think what you're doing is kind of bad-T., architecturallyplacing the temporary files in your current directory. Might be you're running out of space there? Or is that current directory on tmpfs or similar?

TL;DR: out-of-memory-killer kills sort. Because sort tool, trying to limit its RAM usage, assumes temporary files are not using RAM. On most modern Linuxes, they are. Recommendation: Use a different tool.

You can "help" sort around that by changing the temporary directory it uses. Gonna let you look that up yourself, because I think what you're doing is kind of bad, architecturally.

TL;DR: out-of-memory-killer or running out of disk space for temporary files kills sort. Recommendation: Use a different tool.

You can "help" sort around that by changing the temporary directory it uses. You're already doing that, -T., placing the temporary files in your current directory. Might be you're running out of space there? Or is that current directory on tmpfs or similar?

Source Link
Marcus Müller
  • 52.9k
  • 4
  • 80
  • 123
Loading