Motion Magnitude

[repers]

Hi there, I was intrigued reading your work and thought the idea of using a wavelet transform was interesting. I have a question about the methodology you use to obtain the motion magnitude values you present in Table V/Fig.11
When you use a flow network to estimate the flow you get a tensor in the shape (batch, 2, h, w). To find the mean over the entire set, do you find the mean for each flow tensor e.g. torch.mean(flow) and then do that for each input image. Or do you do something else like concatenating the tensors and then finding the mean along the dimension of concatenation?

Thanks

[ltkong218]

Thanks for your interest. I calculate the flow magnitude for each pixel in the image and get the flow magnitude map of shape (batch, 1, h, w), then the entire dataset is traversed to get (batch_total, 1, h, w). Finally, the mean value is calculated over (batch_total, 1, h, w). If the image resolution is relatively high, to save memory, I will downsample the flow magnitude map to (batch, 1, h//s, w//s) in each iteration, where s is a scale factor. Finally, the mean value is calculated over (batch_total, 1, h//s, w//s).

[repers]

Thanks for your response, it's much appreciated
To calculate the flow magnitude, do you take the norm using torch.linalg.norm(x, dim=1, ord=0) or by flow magnitude you refer to the estimated forward and backward flows?

This is what I'm doing now:

    x=0
    for batch_idx, (frame0, frame1, frame2) in enumerate(val_loader):

        frame0, frame2 = frame0.cuda(), frame2.cuda()
        tenFlow = run.estimate(frame0,frame2) # Estimate optical flow
        b,c,h,w = tenFlow.shape
        meanFlow = torch.abs(tenFlow[:,0,:,:].view(b,1,h,w)) # Only take forward flow and view to (batch, 1, h, w)
        if batch_idx == 0:
            x = meanFlow
        else:
            x = torch.cat([x,meanFlow],dim=0) # Final output should be (batch_total, 1, h, w)
    mean=torch.mean(x) 
    return mean`

Can you please let me know what I'm doing incorrectly here. I feel better understanding the motion in a sequence can better help with understanding performance of networks

[repers]

Hi there, can you please let me know how you calculate the motion magnitude. I think the rest should be achievable if magnitude is calculated correctly as issues can be figured better figured out

[ltkong218]

Hi, I was on vacation some time ago. I use the following function to calculate the motion magnitude.

def get_magnitude(flow):
    return (flow[:, 0, :, :] ** 2 + flow[:, 1, :, :] ** 2) ** 0.5

You can try it.

[repers]

Thanks a lot for this! I do get values that are now like your paper, really appreciate your help For ATD12K, the standard deviation of 32.3 which is much higher than the mean 19.4. So does it mean the range of motions in the set is varies a lot in each sequence? Generally a large standard deviation indicates large differences in characteristics for each sequence?

…

________________________________ From: Tone ***@***.***> Sent: 05 October 2023 03:18 To: ltkong218/WaveletVFI ***@***.***> Cc: repers ***@***.***>; Author ***@***.***> Subject: Re: [ltkong218/WaveletVFI] Motion Magnitude (Issue #1) Hi, I was on vacation some time ago. I use the following function to calculate the motion magnitude. def get_magnitude(flow): return (flow[:, 0, :, :] ** 2 + flow[:, 1, :, :] ** 2) ** 0.5 You can try it. — Reply to this email directly, view it on GitHub<#1 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AOI2BHMQ4AYFCEJWKVVQKHDX5YKHJAVCNFSM6AAAAAA5DLNDNCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBXHEZTOMJVGE>. You are receiving this because you authored the thread.Message ID: ***@***.***>