Deleting Twilights in BAStag

[AmelieRobertoCharron]

Hello everyone, 

How do you delete twilights using the preprocessLight function in BAStag?

The help file indicates that at stage four you should be able to use the 'd' to toggle deletions. I have found that this changes the color of the sunrise or sunset from orange or blue to gray, but when I show the results of my defined twilights against the raw data that they appear again.

I'm certain it's an easy fix, I'm just unsure how to do it. 

Thanks for your help!  
Amélie

[AmelieRobertoCharron]

Hello again, 

I ran the following code to preprocess with BAStag:

# Set threshold

threshold = log(4.5)

# Preprocess in BAStag

twl

                       lmax = 12)

 

In the preprocessLight function I deleted inaccurate twilights with the 'd' key in the four stage which caused them to turn grey. I then moved on and saved my results. 

# Save result as csv

write.csv(twl, paste("twl.csv", sep = "?/"), row.names = F)

 

# Converts twilights into appropriate format

twl$Twilight

                           tz= "GMT")

 

# Account for storing maximum values for each 5 minute period. 

# Adjust twilight data since this tag records maximum values over time (in this 

# case every 5 minutes, 60*5)

 

twl

And to view the results of the preprocessing I ran this: 
 

# Show results of defining twilights against raw light data

# Offset of 18 puts the night in the middle (so the night isn't cut off)

 

offset = 18

 

lightImage(d.lux, offset = offset, zlim = c(0,12))

 

tsimagePoints(twl$Twilight, offset = offset, pch = 16, cex = 0.5, 

              col = ifelse(twl$Rise, "dodgerblue", "firebrick"))

When I go to view the results all of the twilights I deleted are still present. 

Does anyone know how to delete inaccurate twilights? 

Thanks, 
Amélie

[ebridge]

I've not looked, but I wonder if the "deleted" twilights are actually just flagged somehow and not actually deleted. Can you show us some of the actual data in twl?

[AmelieRobertoCharron]

 Hi Eli, 

Thank you for the response. 

I was trying to delete transitions that using the preprocessLight function in hopes of cleaning up my data to run it in FLightR. 

I deleted transitions in preprocessLight by clicking the "d" key in the fourth step, and recorded by hand which transitions I deleted, so as to have a reference after to see whether specific transitions were removed. When I looked at the results from the preprocessLight function, none of the transitions I deleted were gone, they were all still present. Below I have included some twilight data, the records in red I attempted to delete and they are still present. 
 

Twilight Rise Deleted Marker Inserted Twilight3 Marker3

15/06/2014 5:02 FALSE FALSE 0 FALSE 15/06/2014 5:02 0

15/06/2014 10:21 TRUE TRUE 0 FALSE 15/06/2014 10:21 0

16/06/2014 5:07 FALSE FALSE 0 FALSE 16/06/2014 5:07 0

16/06/2014 10:11 TRUE FALSE 0 FALSE 16/06/2014 10:11 0

17/06/2014 4:52 FALSE FALSE 0 FALSE 17/06/2014 4:52 0

17/06/2014 10:21 TRUE TRUE 0 FALSE 17/06/2014 10:21 0

18/06/2014 4:57 FALSE FALSE 0 FALSE 18/06/2014 4:57 0

18/06/2014 10:01 TRUE FALSE 0 FALSE 18/06/2014 10:01 0

19/06/2014 5:07 FALSE FALSE 0 FALSE 19/06/2014 5:07 0

19/06/2014 10:01 TRUE FALSE 0 FALSE 19/06/2014 10:01 0

20/06/2014 4:47 FALSE TRUE 0 FALSE 20/06/2014 4:47 0

20/06/2014 10:16 TRUE FALSE 0 FALSE 20/06/2014 10:16 0

21/06/2014 5:07 FALSE FALSE 0 FALSE 21/06/2014 5:07 0

21/06/2014 10:16 TRUE FALSE 0 FALSE 21/06/2014 10:16 0

22/06/2014 5:17 FALSE FALSE 0 FALSE 22/06/2014 5:17 0

22/06/2014 10:05 TRUE FALSE 0 FALSE 22/06/2014 10:05 0

23/06/2014 5:12 FALSE FALSE 0 FALSE 23/06/2014 5:12 0

23/06/2014 10:01 TRUE FALSE 0 FALSE 23/06/2014 10:01 0

24/06/2014 5:07 FALSE FALSE 0 FALSE 24/06/2014 5:07 0

24/06/2014 10:05 TRUE FALSE 0 FALSE 24/06/2014 10:05 0

25/06/2014 4:47 FALSE TRUE 0 FALSE 25/06/2014 4:47 0

25/06/2014 10:40 TRUE TRUE 0 FALSE 25/06/2014 10:40 0

26/06/2014 5:07 FALSE FALSE 0 FALSE 26/06/2014 5:07 0

26/06/2014 10:11 TRUE FALSE 0 FALSE 26/06/2014 10:11 0

27/06/2014 4:58 FALSE FALSE 0 FALSE 27/06/2014 4:58 0

27/06/2014 10:11 TRUE FALSE 0 FALSE 27/06/2014 10:11 0

I assume it's possible to delete transitions, as others have reported that they have deleted unrealistic transitions, but I can't seem to make it work in preprocessLight. As an alternative I can export the twilight times and delete them by hand in excel and reimport it into R, but that seems much more complicated than simply doing it in preprocessLight.

If you have any advice I'd really appreciate it!
Thanks again, 
Amélie 
 

[AmelieRobertoCharron]

I should also add that I have tried removing both sunrise and sunset times for the same day or light curve when there either are unrealistic, and have found that they also aren't deleted when I look at the twl data.

Thanks, 
Amélie 

[ebridge]

Just subset the data:

 

twl_d

 

or something like that.

 

But I'd recommend carrying on with the full data set if possible. If you publish or share the data with lines deleted your work is not really repeatable.

[AmelieRobertoCharron]

Hi Eli, 

Thanks for your help! Subsetting the data worked. 

And thanks for your recommendations. I will keep as many transitons as possible.

Thanks again,
Amélie

[Hendrik]

HI Eli and Amelie,

 

I just had a look at this post and have a question myself. As previously discussed, BasTag only marks deleted twillights as "deleted", but keeps them in the output file. Once converted into TAGS format, these twilights are marked as excluded = TRUE. Now my question is, will FlightR automatically exclude these twilights marked as "excluded" from the analysis? 

 

Thanks a lot for clarifying and have a good start in 2017!

 

All the best,

 

Hendrik

[Eldar]

Now my question is, will FlightR automatically exclude these twilights marked as "excluded" from the analysis? 

 Yes it will.

     Eldar