From 9d0302c367f5186592674434f4a0d700d7fdc3ed Mon Sep 17 00:00:00 2001 From: Your Name Date: Sun, 28 Jun 2026 14:02:14 -0400 Subject: [PATCH] data model added --- tl/data_model.org | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 tl/data_model.org diff --git a/tl/data_model.org b/tl/data_model.org new file mode 100644 index 0000000..689956a --- /dev/null +++ b/tl/data_model.org @@ -0,0 +1,19 @@ +#+title: Data Model + +a mark-group is the fundamental datastructure of this app. the whole scene graph is composed of them. we use this same structure to represent the main timeline, the clips within the timeline, and the annotations on the timeline. a mark-group is just an ordered list of marks with a little bit of extra data hanging off them depending on the type. it also has a parent: the mark-group context it belongs to. + +a mark can represent an instant or a range of time, and that instant or range can be defined in terms of frames (relative to the current timeline on the timeline stack) or frames w/r/t to other mark-groups and these can be mixed and matched. each mark can also optionally specify a target video track (video tracks and clips are initially sourced from an initial otio file). thus, the main timeline is a mark group with one mark: the start and end timestamp. a clip which belongs to that main timeline is a mark group with one mark: the start and end timestamps (within the main timeline) AND the video track to which it belongs. its parent is the main timeline (or should it have no parent? since these are the fundamental units maybe it's ok if they are seen everywhere. basically we have to make a choice: do we copy the clips into new contexts? like if we do it such that a clip has a parent, and the annotation on that clip has the same parent, they are siblings -- what does it mean to expand the annotation by putting it on the timeline stack w/r/t the underlying sibling clips, then? really in that case, the clip should become a child of the annotation, but it also needs to be the child of the main timeline as well, and any other sub annotation...seems better to have them parentless). an annotation is a mark group which conceptually represents a point or points of interest with optional commentary, though it looks not substantially different than a timeline or clip in the data structure, and really it's so flexible it could represent whole re-edits of clips. its marks can also be timestamps in the context of some timeline, or they can be the start and end frames of a given clip, defined relative to the clip, or defined relative to another annotation, or any combination thereof, in any amount, and with any combination of ranges and instants. (should an instant really just be a range of one frame? rather than its own separate thing?) + +because an annotation is just a mark-group, and a timeline is just a mark group, any annotation can be pushed onto the timeline-stack, replacing the main timeline. the clips within the marks in the mark group are laid end to end to form one continuous duration, a new timeline. the cool thing here is that you can now annotate within the context of this annotation. so if we are inside annotation A, annotation A is the :parent of our new Annotation B. if annotation B uses absolute timestamp marks, they are relative to the annotation A timeline, not the main timeline. and if annotation B uses clip based timestamps, they can only reference the frames of the underlying clip which are within range of the annotation (annotation A, the parent, may have start half way through the clip at the beginning, and end half way through the clip at the end). and then you can push annotation B onto the timeline stack, annotate within that, and on and on. + +something about mark groups to note is that marks need not be defined in order. if the main timeline has clip A, B, and C laid end to end, an annotation X can have marks [[clipC[0], clipC[-1]], [clipA[0], clipA[-1]], [clipB[0], clipB[-1]]. -1 represents last available frame of the clip (note here that available frame may differ from absolute last frame of the underlying clip, because the annotation context we're in could cut off half the clip, for example). in this example, we have totally rearranged the clips into a timeline B C A, end to end. and you can also imagine we can cut clips in half, interleave them, repeat them and so on. + +hmm here's a struggle though. let's say i have interleaved half of A with half of B in an annotation which i pushed onto the timeline stack: + +marks: [[clipB[20], clipB[40]], [clipA[0], clipA[20]], [clipB[0], clipB[20]] [clipA[20], clipA[40]]] + +we want to be able to mark these sub clips independently for a new annotation Y, right? but these are just 2 root clips that became 4. so how can we define annotation Y with respect to any of these 4 clips? we don't want to use absolute timeline time, but we also don't want to use absolute clip time because our marks can cross between the first sub clip (b 20 - 40) and the second (a 0 to a 20), so clip time means nothing here. we also need to remember that the parent timeline can clear out its marks at will. so we can definitely orphan annotations - that's ok, that's a UI concern we can display warnings for and just grey out basically (drop to bottom of annotation list, for example, with a warn emoji and let the user edit to specify its marks again, even warn on which marks are broken, and if at least one mark is still valid still display it there). so it rly seems like an annotation SHOULD create synthetic subclips which point at the raw clips so that we can use the synthetic subclips as our targets. but they need to be stable identities, serializable/deserializable. + +this brings us to playing. since right now there is only one source video file, we need to be able to seek to arbitrary frames. we should have a playhead stack, or each annotation can maintain its own playhead that gets used when its the top of the timeline stack. when we hit play, in the example above of annotation X, we find the clip under the playhead of the current timeline on the timeline stack, find where we are relative to the root clip, and use that root clip's frame to seek to that frame in the source video file. and start playing. when the playhead advances and the clip under it has changed, we look to see if there is a difference between that absolute frame in the source file (computed from playhead position and the clip) and where we would expect to be in the file. if it differs, we seek to that absolute frame and continue playing, checking every frame if we need to seek. does that make sense? this should be an extremely simple function. because we're rendering the timeline already, we should already know which frame we should be on without even recursing, i think? +# how do we determine which tracks are included when we zoom into each annotation? for now it should just be if a clip is within the ranges of the mark-group, its track is included in the annotation. +# automatically scroll to bottom-most track in mark group range when we hit the start mark? but what if it's massively spread out. maybe not then. scrolling should be an option turned on. thats ok. make it explicit.