CrowdSky Module

CrowdSky time-block stacking and server sync.

Chunks (local stacking)

CrowdSky time-block stacking for citizen-science time-domain astronomy.

This module automates the production of fixed-duration stacked chunks from a Seestar observation session. Raw sub-frames are grouped into clock-aligned time blocks (default 15 minutes), and each block is batch-stacked independently. Output files are renamed to a CrowdSky_* naming convention that encodes the block boundary timestamp and filter, enabling deterministic idempotent re-runs.

Typical workflow:

from seestarpy import crowdsky

# 1. See what's on the Seestar
crowdsky.list_targets()

# 2. Preview unstacked blocks for one target
crowdsky.stack_blocks("M 81", dry_run=True)

# 3. Stack one target
crowdsky.stack_blocks("M 81")

# 4. Or stack everything at once
crowdsky.stack_all(dry_run=True)   # preview
crowdsky.stack_all()               # run
seestarpy.crowdsky.chunks.compute_chunk_key(block_start, ra_deg, dec_deg)

Build a CrowdSky chunk key from a local block start and sky position.

Returns a string like "20250115.78_HP049152".

seestarpy.crowdsky.chunks.filter_covered_blocks(blocks, covered)

Remove blocks whose UTC chunk key is in the covered set.

Parameters:

  • blocks (dict) – As returned by group_frames_into_blocks().

  • covered (set[tuple[str, str, str]]) – (chunk_str, exposure, filter) tuples.

Returns:

Sorted list of uncovered block dicts.

Return type:

list[dict]

seestarpy.crowdsky.chunks.find_unstacked_blocks(target, block_minutes=15)

Find time blocks of raw frames that have not yet been batch-stacked.

Scans the raw light frames for target, groups them into clock-aligned time blocks of block_minutes duration, and returns blocks that don’t yet have a matching CrowdSky_* output file. Blocks are sub-grouped by (exposure, filter) so that mixed-mode observations produce separate stacks with compatible frames.

Parameters:

  • target (str) – Target name as it appears in folder names, e.g. "M 81".

  • block_minutes (int) – Block duration in minutes. Default 15. Blocks are aligned to clock boundaries (e.g. :00, :15, :30, :45).

Returns:

Sorted list of unstacked sub-blocks. Each dict contains:

  • block_start (datetime): Start of the time block.

  • block_end (datetime): End of the time block.

  • exposure (str): e.g. "20.0s".

  • filter (str): e.g. "LP".

  • files (list[str]): Sorted raw filenames in this sub-block.

  • frame_count (int): Number of files.

Return type:

list[dict]

Examples

>>> from seestarpy import crowdsky
>>> blocks = crowdsky.find_unstacked_blocks("M 81")
>>> for b in blocks:
...     print(f"{b['block_start']:%H:%M}  {b['frame_count']} frames")
seestarpy.crowdsky.chunks.group_frames_into_blocks(parsed_frames, block_minutes=15)

Group parsed frame dicts into clock-aligned time blocks.

Parameters:

  • parsed_frames (list[dict]) – Each dict must have at least keys filename, exposure, filter, datetime (as returned by parse_light_filename() with filename added).

  • block_minutes (int) – Block duration in minutes. Default 15.

Returns:

Keyed by (block_start, exposure, filter) tuples. Values are dicts with keys: block_start, block_end, exposure, filter, files (sorted list), frame_count.

Return type:

dict

seestarpy.crowdsky.chunks.list_targets()

List observation targets that have raw sub-frames available for stacking.

Scans the Seestar’s MyWorks folder for target/target_sub folder pairs, indicating targets with both stacked outputs and raw sub-frames.

Returns:

Each dict contains:

  • target (str): Target name.

  • raw_files (int): Number of files in the _sub folder.

  • stacked_files (int): Number of files in the main folder.

Return type:

list[dict]

Examples

>>> from seestarpy import crowdsky
>>> crowdsky.list_targets()
[{'target': 'M 81', 'raw_files': 1052, 'stacked_files': 1}]
seestarpy.crowdsky.chunks.local_dt_to_chunk_str(dt_local)

Convert a naive local datetime to a YYYYMMDD.CC UTC chunk string.

CC is the 15-minute chunk index (0–95) within the UTC day.

seestarpy.crowdsky.chunks.parse_coverage_from_filenames(filenames)

Extract coverage tuples from CrowdSky_* filenames.

Handles both the current format (with HEALPix pixel) and the legacy format (with local timestamp).

Parameters:

filenames (iterable of str) – Filenames to scan for CrowdSky_* patterns.

Returns:

Set of (chunk_str, exposure, filter) tuples representing blocks that are already covered.

Return type:

set[tuple[str, str, str]]

seestarpy.crowdsky.chunks.parse_light_filename(filename)

Parse a raw light frame filename into its components.

Parameters:

filename (str) – e.g. "Light_M 81_20.0s_LP_20260227-225203.fit"

Returns:

Dict with keys target, exposure, filter, datetime. Returns None if the filename doesn’t match the expected pattern.

Return type:

dict or None

Examples

>>> parse_light_filename("Light_M 81_20.0s_LP_20260227-225203.fit")
{'target': 'M 81', 'exposure': '20.0s', 'filter': 'LP',
 'datetime': datetime(2026, 2, 27, 22, 52, 3)}
seestarpy.crowdsky.chunks.purge_crowdsky_stacks(folder=None)

Delete all CrowdSky_* files from observation folders on the Seestar.

Parameters:

folder (str or None) –

  • A specific folder name (e.g. "M 81") — purge only that folder.

  • "all" or None — scan every non-_sub folder and purge all.

Returns:

{"folders_scanned": int, "files_deleted": int, "deleted": [str]}

Return type:

dict

seestarpy.crowdsky.chunks.stack_all(block_minutes=15, min_exptime=240, dry_run=False)

Find and batch-stack all unstacked time blocks for every target.

Discovers targets via list_targets(), then calls stack_blocks() for each one sequentially. If one target fails, the error is recorded and processing continues with the next target.

Parameters:

  • block_minutes (int) – Block duration in minutes. Default 15.

  • min_exptime (float) – Minimum total effective exposure in seconds. Default 240.

  • dry_run (bool) – If True, preview what would be stacked without doing it.

Returns:

Summary with keys: targets_processed, targets_with_work, total_blocks_stacked, total_blocks_failed, total_blocks_skipped, per_target.

Return type:

dict

Examples

>>> from seestarpy import crowdsky
>>> crowdsky.stack_all(dry_run=True)
>>> result = crowdsky.stack_all()
seestarpy.crowdsky.chunks.stack_blocks(target, block_minutes=15, min_exptime=240, dry_run=False)

Find and batch-stack all unstacked time blocks for a target.

This is the main CrowdSky orchestrator. It discovers unstacked blocks via find_unstacked_blocks(), filters by minimum exposure time, and submits sequential batch stack jobs to the Seestar.

After each successful stack the output file is renamed from the firmware’s DSO_Stacked_* format to CrowdSky_* with the block boundary timestamp and filter encoded in the filename. This makes re-runs fully idempotent.

Parameters:

  • target (str) – Target name, e.g. "M 81".

  • block_minutes (int) – Block duration in minutes. Default 15.

  • min_exptime (float) – Minimum total effective exposure (frame_count * exposure_seconds) in seconds for a block to be worth stacking. Default 240.

  • dry_run (bool) – If True, print what would be stacked without actually doing it.

Returns:

Summary with keys: target, blocks_stacked, blocks_failed, blocks_skipped, results.

Return type:

dict

Examples

>>> from seestarpy import crowdsky
>>> result = crowdsky.stack_blocks("M 81", dry_run=True)
>>> result = crowdsky.stack_blocks("M 81")

Server (web API)

CrowdSky web API client for uploading and downloading stacked FITS files.

This module wraps the CrowdSky server’s HTTP Basic Auth endpoints:

  • GET /api/my_stacks.php — list user’s uploaded stacks

  • POST /api/upload_stack.php — upload a pre-stacked FITS file

  • GET /api/download_stack.php — download a stacked FITS by chunk_key

  • POST /api/raw_upload.php — batch-upload raw sub-frames (start / file / finalize)

Credentials can be set via environment variables (CROWDSKY_USERNAME, CROWDSKY_PASSWORD) or at runtime with set_credentials().

Example:

from seestarpy.crowdsky import server

server.set_credentials("alice", "s3cret")

# List existing stacks
stacks = server.list_stacks()

# Upload a pre-stacked result
server.upload_stack("CrowdSky_38_M81_20.0s_LP_20260227-224500.fit")

# Upload raw sub-frames
token = server.raw_start_session()
for path in raw_fits_files:
    server.raw_upload_file(token, path)
result = server.raw_finalize(token)

# Re-upload raw files, replacing any existing stack for the same chunk-key
token = server.raw_start_session()
for path in raw_fits_files:
    server.raw_upload_file(token, path)
result = server.raw_finalize(token, overwrite=True)
seestarpy.crowdsky.server.download_stack(chunk_keys, dest='.')

Download stacked FITS files from the CrowdSky server by chunk key.

Parameters:

  • chunk_keys (str or list[str]) – One or more chunk keys to download.

  • dest (str or Path) – Destination directory. Defaults to current directory.

Returns:

List of local file paths that were downloaded.

Return type:

list[Path]

seestarpy.crowdsky.server.list_stacks(object_name=None)

List the user’s uploaded stacks on the CrowdSky server.

Parameters:

object_name (str, optional) – Filter results by object name (passed as ?object= query param).

Returns:

Each dict contains server-side metadata such as chunk_key, object_name, n_frames, etc.

Return type:

list[dict]

seestarpy.crowdsky.server.raw_finalize(session_token, scrub_location=False, overwrite=False)

Finalize a raw-upload session and create stacking jobs.

Groups all uploaded files by chunk_key + telescope_id and inserts stacking_jobs rows. The worker will pick them up and stack them.

Parameters:

  • session_token (str) – Token returned by raw_start_session().

  • scrub_location (bool) – If True, strip SITELONG/SITELAT from the stacked output FITS and database record. Default False.

  • overwrite (bool) – If True, delete any existing stacked_frames row (and its u:cloud files) for each duplicate chunk-key before creating the new stacking job. If False (default), duplicate chunk-keys are skipped and returned in the skipped list.

Returns:

Server response with keys:

  • ok (bool)

  • jobs (int): number of stacking jobs created

  • job_ids (list[int]): IDs of the created jobs

  • chunks (list[str]): chunk-keys that were queued

  • skipped (list[str]): chunk-keys skipped because a stack already existed (only populated when overwrite is False)

Return type:

dict

Raises:

RuntimeError – If the server returns an error.

seestarpy.crowdsky.server.raw_start_session()

Create a new raw-upload session on the server.

Returns:

The session_token for this upload session, which must be passed to raw_upload_file() and raw_finalize().

Return type:

str

Raises:

RuntimeError – If credentials are not set or the server returns an error.

seestarpy.crowdsky.server.raw_upload_file(session_token, fits_path)

Upload one raw FITS sub-frame to an open upload session.

The file must have a valid DATE-OBS FITS header keyword; the server returns a 400 error if it is absent.

Parameters:

  • session_token (str) – Token returned by raw_start_session().

  • fits_path (str or Path) – Local path to the raw .fit / .fits sub-frame.

Returns:

Server response with keys ok, filename, chunk_key, date_obs.

Return type:

dict

Raises:

  • FileNotFoundError – If fits_path does not exist.

  • RuntimeError – If the server returns an error for this file.

seestarpy.crowdsky.server.set_base_url(url)

Override the CrowdSky server URL (for testing or staging).

Parameters:

url (str) – Base URL without trailing slash, e.g. "https://staging.crowdsky.example.com".

seestarpy.crowdsky.server.set_credentials(username, password)

Set CrowdSky login credentials for the session.

Parameters:

  • username (str) – CrowdSky account username.

  • password (str) – CrowdSky account password.

seestarpy.crowdsky.server.upload_all_stacks(target=None, local_dir=None, dest='.', skip_existing=True, dry_run=False)

Upload all CrowdSky-ready stacks to the CrowdSky server.

Finds CrowdSky_*.fit files either on the Seestar’s eMMC or in a local directory, and uploads each one via upload_stack().

Parameters:

  • target (str, list[str], or None) –

    Which observation folders to scan on the Seestar:

    • A target name (e.g. "M 81") — that folder only.

    • A list of names — those folders.

    • None or "all" — every non-_sub folder.

    Ignored when local_dir is set.

  • local_dir (str, Path, or None) – If given, scan this local directory (recursively) for CrowdSky_*.fit files instead of downloading from the Seestar.

  • dest (str or Path) – Local directory where files downloaded from the Seestar are saved. Files are kept after upload. Default ".". Ignored when local_dir is set.

  • skip_existing (bool) – If True (default), query the server first and skip files whose chunk key has already been uploaded.

  • dry_run (bool) – If True, list what would be uploaded without transferring data.

Returns:

Summary with keys folders_scanned, files_uploaded, files_skipped, files_failed, uploaded, skipped, failed.

Return type:

dict

Examples

Upload stacks from a specific target on the Seestar:

>>> from seestarpy.crowdsky import server
>>> server.set_credentials("alice", "s3cret")
>>> server.upload_all_stacks("AM Leo", dest="./downloads")

Upload from a local directory (no Seestar needed):

>>> server.upload_all_stacks(local_dir="./my_stacks")

Preview without uploading:

>>> server.upload_all_stacks(dry_run=True)
seestarpy.crowdsky.server.upload_stack(fits_path, thumbnail=None, n_frames_input=None, n_frames_aligned=None, date_obs_start=None, date_obs_end=None, scrub_location=None)

Upload a stacked FITS file to the CrowdSky server.

Parameters:

  • fits_path (str or Path) – Local path to the .fit / .fits file.

  • thumbnail (str or Path, optional) – Local path to a thumbnail image (JPEG/PNG).

  • n_frames_input (int, optional) – Total number of input sub-frames.

  • n_frames_aligned (int, optional) – Number of frames that were successfully aligned.

  • date_obs_start (str, optional) – ISO 8601 timestamp of the first sub-frame.

  • date_obs_end (str, optional) – ISO 8601 timestamp of the last sub-frame.

  • scrub_location (int, optional) – If 1, strip SITELONG/SITELAT from the output FITS and DB.

Returns:

Server response with keys ok, job_id, chunk_key, status ("queued"). The file is queued for worker post-processing rather than stored directly.

Return type:

dict

Raises:

FileNotFoundError – If fits_path does not exist.