Skip to main content
some fiormatting
Source Link
Tulains Córdova
  • 39.6k
  • 13
  • 102
  • 157

In our system a media file can have several states: invalid, empty, uploading, ingesting, ready… Depending

  • invalid
  • empty
  • uploading
  • ingesting
  • ready…

..depending of the state, information about the media areis accessed in different ways. For example, during upload the file name is on a temporary upload info JSON file on the filesystem. When the object is ready this file name would be guessed by looking at a specific place on the filesystem (namely under an directory named after the ID of the media).:

  • During upload the file name is on a temporary upload info JSON file on the filesystem.
  • When the object is ready this file name would be guessed by looking at a specific place on the filesystem (namely under an directory named after the ID of the media).

Same goes for the "progress" attribute of a media : during upload "progress" represents the downloaded bytes / file size. During the ingestion step, the progress represents the current step number / total number steps (thumbnailing, post-processing, etc).

  • During upload "progress" represents the downloaded bytes / file size.
  • During the ingestion step, the progress represents the current step number / total number steps (thumbnailing, post-processing, etc).

The serialized state of a media is used by the UI to feed information back to the user.

A MediumStateFactory tries a bunch of AbstractMediumState derived classes (MediumStateReady, MediumStateUploading, etc) until one qualifies to handle the current state of the media.

Turns out that for tiny files the state can change during serialization: an MediumStateUploading can be built at the beginning of the serialization, but as the state of the media changes, by the time the actual serialization is done it doesn't represent the media anymore, leading to errors. For example, a MediumStateUploading expects to retrieve the media size from the temporary JSON.

Currently we are restarting the serialization entirely if we catch an exception during the process. This is rather brute force and quite ugly.

What would be the best way to serialize objects that can mutate during the serialization ?What would be the best way to serialize objects that can mutate during the serialization process?

In our system a media file can have several states: invalid, empty, uploading, ingesting, ready… Depending of the state, information about the media are accessed in different ways. For example, during upload the file name is on a temporary upload info JSON file on the filesystem. When the object is ready this file name would be guessed by looking at a specific place on the filesystem (namely under an directory named after the ID of the media).

Same goes for the "progress" attribute of a media : during upload "progress" represents the downloaded bytes / file size. During the ingestion step, the progress represents the current step number / total number steps (thumbnailing, post-processing, etc).

The serialized state of a media is used by the UI to feed information back to the user.

A MediumStateFactory tries a bunch of AbstractMediumState derived classes (MediumStateReady, MediumStateUploading, etc) until one qualifies to handle the current state of the media.

Turns out that for tiny files the state can change during serialization: an MediumStateUploading can be built at the beginning of the serialization, but as the state of the media changes, by the time the actual serialization is done it doesn't represent the media anymore, leading to errors. For example, a MediumStateUploading expects to retrieve the media size from the temporary JSON.

Currently we are restarting the serialization entirely if we catch an exception during the process. This is rather brute force and quite ugly.

What would be the best way to serialize objects that can mutate during the serialization ?

In our system a media file can have several states:

  • invalid
  • empty
  • uploading
  • ingesting
  • ready…

..depending of the state, information about the media is accessed in different ways. For example:

  • During upload the file name is on a temporary upload info JSON file on the filesystem.
  • When the object is ready this file name would be guessed by looking at a specific place on the filesystem (namely under an directory named after the ID of the media).

Same goes for the "progress" attribute of a media :

  • During upload "progress" represents the downloaded bytes / file size.
  • During the ingestion step, the progress represents the current step number / total number steps (thumbnailing, post-processing, etc).

The serialized state of a media is used by the UI to feed information back to the user.

A MediumStateFactory tries a bunch of AbstractMediumState derived classes (MediumStateReady, MediumStateUploading, etc) until one qualifies to handle the current state of the media.

Turns out that for tiny files the state can change during serialization: an MediumStateUploading can be built at the beginning of the serialization, but as the state of the media changes, by the time the actual serialization is done it doesn't represent the media anymore, leading to errors. For example, a MediumStateUploading expects to retrieve the media size from the temporary JSON.

Currently we are restarting the serialization entirely if we catch an exception during the process. This is rather brute force and quite ugly.

What would be the best way to serialize objects that can mutate during the serialization process?

Bumped by Community user
Bumped by Community user
Tweeted twitter.com/StackProgrammer/status/732293708314476545
Source Link
Charles
  • 141
  • 2

Serializing mutating objects

In our system a media file can have several states: invalid, empty, uploading, ingesting, ready… Depending of the state, information about the media are accessed in different ways. For example, during upload the file name is on a temporary upload info JSON file on the filesystem. When the object is ready this file name would be guessed by looking at a specific place on the filesystem (namely under an directory named after the ID of the media).

Same goes for the "progress" attribute of a media : during upload "progress" represents the downloaded bytes / file size. During the ingestion step, the progress represents the current step number / total number steps (thumbnailing, post-processing, etc).

The serialized state of a media is used by the UI to feed information back to the user.

A MediumStateFactory tries a bunch of AbstractMediumState derived classes (MediumStateReady, MediumStateUploading, etc) until one qualifies to handle the current state of the media.

Turns out that for tiny files the state can change during serialization: an MediumStateUploading can be built at the beginning of the serialization, but as the state of the media changes, by the time the actual serialization is done it doesn't represent the media anymore, leading to errors. For example, a MediumStateUploading expects to retrieve the media size from the temporary JSON.

Currently we are restarting the serialization entirely if we catch an exception during the process. This is rather brute force and quite ugly.

What would be the best way to serialize objects that can mutate during the serialization ?