Version 4.1.0

Version 4.1.0 contains 3 important improvements. 2 of them are the obvious answer to several requests I have received during last months. The last one is something I tried to mimic from Flickr website.

These 3 new features are:

  • Incremental uploads among F-Stop runs
  • Ability to displays photos outside photosets
  • Camera Roll like function

Incremental uploads among F-Stop runs

Up to version 4.0.5, F-Stop was able to perform incremental uploads. There was, however, a significant limitation. Incremental upload information about what was successfully uploaded or not, was stored in memory. As memory is volatile, this information was not preserved among F-Stop runs leading to duplicate file uploads when running, for two times in a row, the same folder upload.

Since version 4.1.0, F-Stop stores upload information locally on your computer hard-drive. Before an upload, F-Stop builds a list of files (which need to be uploaded) merging information between the list of files present inside a directory and an upload information store which contains all files already upload for the directory itself. Whenever a file is successfully uploaded, F-Stop appends file path to its upload information store.

Along with file uploads, F-Stop now preserves information about user settings applied to a folder, such as name, description, tags, etc.. Fill a batch with a folder (with stored settings), and F-Stop automatically restores them without user intervention.

As this functionality increases disk space and processing power, you can opt-in/out within F-Stop preferences.

You can find more information about incremental uploads in this post.

Ability to displays photos outside photosets

Another functionality I have added is related to photosets management. Some users requested a way to figure out which photos are not stored inside a photoset in a way similar to Flickr website organizer.

So now, when you are browsing a photoset, you can see a small wrench icon in the toolbar. Click it and open the right sidebar where you see 2 icons (on top). The first one shows comments for this photoset, the other one photos not belonging to any photoset.

Photos Not In Sets

From the sidebar you can drag&drop one or multiple photos over the first item of the set.

It’s important to know that this feature uses a specific Flickr API named flickr.photos.getNotInSet. This api lacks a way to specify a referring photoset. In other words, it returns photos not belonging to any photosets. This limitation can lead to the following scenario:

You are browsing photoset named “A”. Photoset “A” contains photo1 and photo2 while your photo stream contains photo1, photo2, photo3 and photo4. Photo4 belongs to photoset named “B”.

While browsing Photoset “A”, F-Stop will call flickr.photos.getNotInSet. This API will ONLY return photo3 and it will NOT return photo4. This because photo3 is the only photo which doesn’t belong to any photoset.

It would be nice to specify which photoset are you interested in when calling this api. In this way, F-Stop could say Flickr: “Hey, gimme all photos which don’t belong to photoset A”. That would fix the above scenario. I hope Flickr will add this capability soon.

Camera Roll

Flickr has added a new functionality to its website named Camera Roll. Camera Roll is in beta stage and it’s difficult to predict all its features but, at the time of writing, it seems a way to have your photos organized per date (taken or uploaded). It sports a nice tiled interface with a left timeline used to filter events in the camera roll. It is nice, fast and very easy to use but its functionality is somehow limited by the lack of the following capabilities:

  • Show camera roll of other users
  • Filter date ranges (very important for huge photosets)
  • Invert the order of time line (Most recent are always on top)

So, I have decided to implement something similar in F-Stop. I admit that my timeline implementation is not as beautiful as the Flickr one but, from a functionality perspective, I believe F-Stop is far ahead.

Camera Roll functionality is accessed using Camera Roll button available while browsing user photo streams (your own or another one)

Camera Roll Button

 

A new view appears

Camera Roll

Clicking the wrench icon on the toolbar, a sidebar appears. From this sidebar you can choose:

  • Camera Roll main ordering field (upload date or taken date)
  • Timeline order
  • Date range

Date range can be null, partial or full.
Null range means that no date filters are applied.
Partial range means that you can specify just one boundary for the range. For example: show camera roll just for photos newer than 2014.
Full range means a completely defined range between 2 dates.

Details about Incremental Uploads

Up to version 4.0.5, F-Stop was able to perform incremental uploads. There was, however, a significant limitation. Incremental upload information about what was successfully uploaded or not, was stored in memory. As memory is volatile, this information was not preserved among F-Stop runs leading to duplicate file uploads when running, for two times in a row, the same folder upload.

Since version 4.1.0, F-Stop overcomes this limitation storing information locally on your hard drive. This is an opt-in/out feature that is activated within F-Stop preferences.

Incremental Uploads Preferences

How it works

To begin, this option has no meaning for single files added to a batch as they are usually added for a specific need. Instead, it works for folders. When this option is enabled, F-Stop will slightly change its behaviour during folders uploads.

Current directory upload process can be simplified as follows: when a batch starts, a new background thread is spawned. This thread performs several administrative tasks such as checking for other running upload threads. Then, if everything is ok, the thread builds a file list of files to upload for each folder present inside the batch itself.

Up to version 4.0.5 this file list was built creating 2 different sets. The first set contained all files stored inside the directory. The second set contained all files already uploaded to Flickr. From the subtraction of set 1 and set 2 F-Stop determined which files needed to be uploaded.

Since version 4.1.0, this process is almost the same but set 2 is stored locally and it’s reread every time a batch starts. Furthermore, this “information store” grows every time a file is successfully uploaded. In this way, it’s always up-to-date with the current batch progress.

N.B. F-Stop will behave as version 4.5.0 and below if “Allows incremental upload” preference is disabled. This preference can be changed at any time but changing this while upload batches are running can have some side effect. Please disable/enable it while no uploads are running

Information stores are placed inside this directory:

~/Library/Containers/com.zamioware.F-Stop/Data/Library/Application Support/com.zamioware.F-Stop/uploadStatus

There a 2 kind of stores:
The first one store is a file with a iue extension used to keep track of uploaded files for each folder. Store file name is a generated name built from the following components:

  • Flickr account nsid
  • MD5 Hash of full directory path
  • Legible directory name

You see that there is a strict correlation between Flickr user name and directory path. In this way, F-Stop can keep track of different uploads of the same folder in multiple accounts. MD5 hash of full path guarantees, instead, that 2 different folders with the same name are not treated as they were the same.

The second information store is a file with sfe extension. This store contains information about user settings applied to a folder (in upload view). This settings are:

Name, Description, Tags, Groups, Photosets, Safety Level, etc.

SFE files are read when a folder is added to a batch so that user settings can be restored.

N.B. F-Stop will read/write SFE only if “Allows incremental upload” preference is enabled. SFE file is stored when at a least one file is successfully uploaded to Flickr.

Managing Incremental Uploads

Incremental uploads can be managed within F-Stop Preferences. To do this, simply click Details button on the right of “Allows incremental upload” preference. When you click details a new window appears:

Incremental Uploads View

 

Within this view, you can see all information stored by F-Stop for incremental uploads. For each folder there are generally 2 different information store identified by “File Type” column. “Folder Information” are SFE files described above. “Files Upload DB” are IUE files described above.

From this interface you can remove files you don’t need anymore. Be careful that, removing “Folder Information” will prevent F-Stop to populate automatically fields for this folder in upload view, while removing “Files Upload DB” will prevent F-Stop to make incremental upload for this folder. In other words, a full upload will run again.

Performance consideration

All tests I have made using my developing machine (Macbook Pro, Early 2011, 2,3 GHz Intel Core i5, Samsung 840EVO SSD Hard Drive, 8GB RAM) show negligible performance impact in incremental upload calculation for a folder with 100.000 files. Something around 3 to 5 seconds every time a batch is started.

My suggestion is to avoid upload of huge directories with hundred of thousand of files.

Caveats

Also with Incremental Upload feature, F-Stop cannot be considered a synchronization utility. This because F-Stop doesn’t keep track of relationship between local file names and remote Flickr photos NSID.

Primary effect of this is that F-Stop will not try to re-upload photos uploaded and subsequently removed from Flickr.  To illustrate better the implication of this, imagine the following scenario:

You upload directory named “A”. Directory A contains file photo1, photo2 and photo3. After a full, completed upload, you remove photo2 from Flickr by mistake. Then you decide to start again upload of directory “A” in the assumption that F-Stop will upload photo2 again.

Well, this is not going to happen! you have to manually re-upload photo2.

Version 4.0.5

This morning F-Stop version 4.0.5 went live. This is a minor release that enables F-Stop to open Flickr links directly. This function is useful when you receive Flickr links via email and you want to open them within F-Stop.

Links supported by F-Stop are the ones generated using share function from Flickr.com as depicted in the following image.

 

FlickrShareLinks

 

N.B. Currently short URLs are NOT supported.

To open a link using F-Stop just press CMD+O or “Open Flickr URL” menu item under File Menu. A simple window will appear.

OpenFlickrURL

 

Paste Flickr URL and click ok. If F-Stop is managing multiple accounts, you can specify the one to use. This is a needed step if photos pointed by the Flickr link are not public and only one specific account has the needed grants to view the photos.