Uploaded file distribution within /system/ folder

Components

Labels

Description

When file uploads are happening a lot on the site, then all files are stored under same /system/... sub-folder. This doesn't pose any problem to webserver to easily access them, however it makes it near to impossible to manage that files manually. Like view/backup/restore individual files, when there are 2000+ files in one directory.

That's why we are proposing uploaded file distribution engine, that will create sub-folders within target upload folder to store uploaded files. Here are some ideas:

[b]1. Filenames/b

a. ability to have Prefix (defined in Field declaration (in unit config), default is empty, DATE-TIME which reserved, but can specify any other symbols)

b. ability to have Ending (defined in Field declaration (in unit config), default is DATE-TIME which reserved, but can specify any other symbols)

DATE = 20110131 (2011/01/31)
TIME = 123401 (12:34:01)

Examples:

  • 20110131-123401_my-custom-filename-whatever.jpg

  • my-custom-filename-whatever_20110131-123401.jpg

or other combinations

This will give us full flexibility in filenames

[b]2. Folders/b

Similar here, give 3 options for each upload field:

a. As is now - All files go into single specified folder.

b. Chronological - YEARMONTH / DAY / (adds 2 sub-levels)

c. Hash - as proposed by Alex (adds 2 sub-levels, see "Additional Information" field for function code)

[b]NOTE:/b
1. Hashed path will be stored along with a filename, since in case of timestamp-based hashing there is no way to determine hash path to a file by it's name.
2. All resized versions will be placed without special structure under /resized/ folder.
3. All code completely/mostly will be located in kUploadFormatter class, so extending that class could give ability to add new storage engine types

Context Information

None

Additional information (do not use)

Hashing function for 2c item:

Unit config field declaration:

Attachments

3

Activity

Alex 
July 25, 2012 at 10:32 AM

Since 5.2.0 version was released.

Alex 
December 15, 2011 at 2:54 PM

It's not possible to keep all resized images in top folder (where they were before) because:

1. full image path is given to ImageHelper::ResizeImage method
2. no way to guess what part of the path is created by storage engine and if it's present at all

Alex 
December 15, 2011 at 2:40 PM

Fix committed to [b]5.2.x branch/b. Commit Message:

Fixes #0001022: Uploaded file distribution within /system/ folder

Alex 
December 15, 2011 at 2:39 PM

Fixed problems:
1. when storage engine used in "images_config.php" file, then images were broken after saving from temp table happened
2. when prefix/suffix plus something else was specified in unit config (e.g. "USER_test"), then "USER" (or any other special keyword) wasn't substituted
3. new method were made "private" resulting inability to override them in extended classes
4. new method PHPDoc comments were not matching actual method declaration
5. spelling errors found
6. storage engine sub-folder wasn't included in unique filename check when non-flash uploader was used

Changes:
1. changed "Ym/d" (e.g. "201112/15") folder format to "Y-m/d" (e.g. "2011-12/15")
2. added "u" in "USER" keyword result (was "_15", now "_u15") to prevent accidental removal by FileHelper::ensureUniqueFilename method
3. exception is raised in case if unsupported storage engine name is specified
4. changed new variable/method namings to fix spelling errors in them
5. added sample code in "widgets_config.php" (otherwise no way for developers to know that we've added something new)
6. constants converted to class format
7. FileHelper::ensureUniqueFilename now also creates folder, that is used to perform check
8. FileHelper::ensureUniqueFilename now also allows a storage folder to be supplied as filename part (will be returned back in result filename too)
9. FileHelper::ensureUniqueFilename changed to use PHP 5.2 version of "pathinfo" function
10. ImageEventHandler::ImageAction method now uses FileHelper::ensureUniqueFilename method instead of it's ValidateFileName outdated copy

Erik S. 
December 13, 2011 at 11:58 AM

Patch uploaded - needs testing

Fixed

Details

Priority

Assignee

Reporter

Developer

Change Log Message

External issue ID

External issue URL

BC Break Details

Reborn methods:

  • "ImageEventHandler::ValidateFileName" into "FileHelper::ensureUniqueFilename"

Story Points

Fix versions

Affects versions

Patch Instructions

Patches must be submitted through Phabricator.

Created March 13, 2011 at 12:21 PM
Updated April 30, 2025 at 10:39 AM
Resolved July 25, 2012 at 10:32 AM