2023.8.27.0

API.JFF: remove PXML attribute for some properties
API.Reddit.Channels: save channel info right after download; replace date providers with default
API.Reddit.SiteSettings: improve 'UpdateToken' function
AutoDownloader: add 'Copy' function
SchedulerEditorForm: add cloning plans
DownloadedInfoForm: add 'Try...Catch' for some functions
DownloadFeedForm: add button to go to custom page
FeedMedia: color typo
GroupParameters: add 'ICopier'

Author: AAndyProgram

Changelog.md

@@ -1,3 +1,22 @@
+# 2023.8.27.0
+
+*2023-08-27*
+
+- Added
+  - **JustForFans**
+  - Advanced download (`Download` - `Download (advanced)`)
+  - Advanced filter (`View` - `Advanced filter`)
+  - Auto downloader: cloning plans
+  - Feed: add button to go to custom page
+  - Special log for non-existent users
+  - Twitter: group 'limit' notifications
+  - Ability to set custom color for subscription users
+  - Other improvements
+- Fixed
+  - Auto downloader: new plan date display bug
+  - Auto downloader: downloading stuck
+  - Minor bugs
+
 # 2023.8.6.0
 
 *2023-08-06*

ProgramScreenshots/SettingsGlobalBasis.png

@@ -11,7 +11,7 @@
 :eu:
 :greece:
 
-A program to download photo and video from [any site](#supported-sites) (e.g. YouTube, YouTube Music, OnlyFans, Reddit, Twitter, Mastodon, Instagram, TikTok, RedGifs, PornHub, XHamster, XVIDEOS, ThisVid, LPSG, Pinterest).
+A program to download photo and video from [any site](#supported-sites) (e.g. YouTube, YouTube Music, OnlyFans, Reddit, Twitter, Mastodon, Instagram, TikTok, RedGifs, JustForFans, PornHub, XHamster, XVIDEOS, ThisVid, LPSG, Pinterest).
 
 **If you like SCrawler, please like the program on [this site](https://alternativeto.net/software/scrawler/about/) and/or [this](https://www.softpedia.com/get/Internet/Download-Managers/Social-networks-crawler.shtml)**
 <!---Do you like this program? Consider adding to my coffee fund by making a donation to show your support. :blush:
@@ -34,6 +34,7 @@ A program to download photo and video from [any site](#supported-sites) (e.g. Yo
   - Redgifs videos (https://www.redgifs.com/);
   - Twitter images and videos, saved (bookmarked) posts;
   - OnlyFans images and videos, saved (bookmarked) posts;
+  - JustForFans images and videos, saved (bookmarked) posts;
   - Mastodon images and videos, saved (bookmarked) posts;
   - Instagram images and videos, tagged posts, stories, saved posts;
   - TikTok videos;
@@ -72,6 +73,7 @@ A program to download photo and video from [any site](#supported-sites) (e.g. Yo
 - **OnlyFans**
 - **Mastodon**
 - **Instagram**
+- JustForFans
 - TikTok
 - RedGifs
 - Pinterest
@@ -90,14 +92,6 @@ A program to download photo and video from [any site](#supported-sites) (e.g. Yo
 
 First, the program downloads the full profile. After the program downloads only new posts. The program remembers downloaded posts.
 
-## Reddit
-
-The program parses user posts, obtain MD5  images hash and compares them with existing ones to remove duplicates. Then the media will be downloaded.
-
-## Other sites
-
-The program parses user posts and compares file names with existing ones to remove duplicates. Then the media will be downloaded.
-
 ## How to request a new site
 
 <!---Read [here](CONTRIBUTING.md#how-to-request-a-new-site) about--->
@@ -128,16 +122,17 @@ The program parses user posts and compares file names with existing ones to remo
   - [Reddit](https://github.com/AAndyProgram/SCrawler/wiki/Settings#reddit)
   - [Twitter](https://github.com/AAndyProgram/SCrawler/wiki/Settings#twitter)
   - [OnlyFans](https://github.com/AAndyProgram/SCrawler/wiki/Settings#onlyfans)
-  - [Mastodon](https://github.com/AAndyProgram/SCrawler/wiki/Settings#Mastodon)
+  - [Mastodon](https://github.com/AAndyProgram/SCrawler/wiki/Settings#mastodon)
   - [Instagram](https://github.com/AAndyProgram/SCrawler/wiki/Settings#instagram)
+  - [JustForFans](https://github.com/AAndyProgram/SCrawler/wiki/Settings#justforfans)
   - [TikTok](https://github.com/AAndyProgram/SCrawler/wiki/Settings#tiktok)
   - [RedGifs](https://github.com/AAndyProgram/SCrawler/wiki/Settings#redgifs)
-  - [YouTube](https://github.com/AAndyProgram/SCrawler/wiki/Settings#YouTube)
+  - [YouTube](https://github.com/AAndyProgram/SCrawler/wiki/Settings#youtube)
   - [Pinterest](https://github.com/AAndyProgram/SCrawler/wiki/Settings#Pinterest)
   - [PornHub](https://github.com/AAndyProgram/SCrawler/wiki/Settings#pornhub)
   - [XHamster](https://github.com/AAndyProgram/SCrawler/wiki/Settings#xhamster)
   - [XVIDEOS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#xvideos)
-  - [ThisVid](https://github.com/AAndyProgram/SCrawler/wiki/Settings#ThisVid)
+  - [ThisVid](https://github.com/AAndyProgram/SCrawler/wiki/Settings#thisvid)
   - [LPSG](https://github.com/AAndyProgram/SCrawler/wiki/Settings#lpsg)
 
 **Full guide you can find [here](https://github.com/AAndyProgram/SCrawler/wiki)**

ProgramScreenshots/SettingsGlobalDesign.png

@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
 ' by using the '*' as shown below:
 ' <Assembly: AssemblyVersion("1.0.*")>
 
-<Assembly: AssemblyVersion("2023.8.17.0")>
-<Assembly: AssemblyFileVersion("2023.8.17.0")>
+<Assembly: AssemblyVersion("2023.8.27.0")>
+<Assembly: AssemblyFileVersion("2023.8.27.0")>
 <Assembly: NeutralResourcesLanguage("en")>

ProgramScreenshots/SettingsSiteJustForFans.png

@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
 ' by using the '*' as shown below:
 ' <Assembly: AssemblyVersion("1.0.*")>
 
-<Assembly: AssemblyVersion("2023.8.17.0")>
-<Assembly: AssemblyFileVersion("2023.8.17.0")>
+<Assembly: AssemblyVersion("2023.8.27.0")>
+<Assembly: AssemblyFileVersion("2023.8.27.0")>
 <Assembly: NeutralResourcesLanguage("en")>

README.md

@@ -1448,7 +1448,7 @@ BlockNullPicture:
                             For i = 0 To _ContentList.Count - 1
                                 data = _ContentList(i)
                                 ProgressPre.Perform()
-                                If (data.Type = UTypes.GIF Or data.Type = UTypes.Picture) Then
+                                If data.Type = UTypes.GIF Or data.Type = UTypes.Picture Then
                                     If data.MD5.IsEmptyString Then
                                         ThrowAny(Token)
                                         eIndx = existingFiles.FindIndex(eFinder)

SCrawler.YouTube/My Project/AssemblyInfo.vb

@@ -30,10 +30,9 @@ Namespace API.JustForFans
         Friend ReadOnly Property UserID As PropertyValue
         <PropertyOption, PXML>
         Friend ReadOnly Property UserHash4 As PropertyValue
-        <PropertyOption(ControlText:="Accept", ControlToolTip:="Header 'Accept'"), PXML>
+        <PropertyOption(ControlText:="Accept", ControlToolTip:="Header 'Accept'")>
         Friend ReadOnly Property HeaderAccept As PropertyValue
-        <PropertyOption, PXML>
-        Friend ReadOnly Property UserAgent As PropertyValue
+        <PropertyOption> Friend ReadOnly Property UserAgent As PropertyValue
         Private Sub UpdateHeader(ByVal HeaderName As String, ByVal HeaderValue As String)
             Select Case HeaderName
                 Case NameOf(HeaderAccept) : If HeaderValue.IsEmptyString Then Responser.Accept = Nothing Else Responser.Accept = HeaderValue

SCrawler.YouTubeDownloader/My Project/AssemblyInfo.vb

@@ -270,13 +270,14 @@ Namespace API.Reddit
                     End With
                     Dim b% = Posts.Count
                     Posts.ListAddList(d.GetNewChannelPosts(), LNC)
-                    If Posts.Count - b > 0 Then CountOfLoadedPostsPerSession.Add(Posts.Count - b)
+                    If Posts.Count - b > 0 Then _Saved = False : CountOfLoadedPostsPerSession.Add(Posts.Count - b)
                     Posts.Sort()
                     LatestParsedDate = If(Posts.FirstOrDefault(Function(pp) pp.Date.HasValue).Date, LatestParsedDate)
                     UpdateUsersStats()
                 End Using
             Catch oex As OperationCanceledException When Token.IsCancellationRequested
             Finally
+                SaveUnsaved()
                 _Downloading = False
             End Try
         End Sub
@@ -344,68 +345,75 @@ Namespace API.Reddit
                 Using x As New XmlFile(f, Protector.Modes.All, False) With {.XmlReadOnly = True, .AllowSameNames = True}
                     x.LoadData()
                     If x.Count > 0 Then
-                        Dim XMLDateProvider As New ADateTime(ADateTime.Formats.BaseDateTime)
                         Dim lc As New ListAddParams(LAP.ClearBeforeAdd)
                         Name = x.Value(Name_Name)
                         ID = x.Value(Name_ID)
                         ViewMode = x.Value(Name_ViewMode).FromXML(Of Integer)(CInt(View.[New]))
                         ViewPeriod = x.Value(Name_ViewPeriod).FromXML(Of Integer)(CInt(Period.All))
                         If FilePosts.Exists Then PostsNames.ListAddList(FilePosts.GetText.StringToList(Of String)("|"), LNC)
-                        LatestParsedDate = AConvert(Of Date)(x.Value(Name_Date), XMLDateProvider, Nothing)
+                        LatestParsedDate = AConvert(Of Date)(x.Value(Name_Date), DateTimeDefaultProvider, Nothing)
                         CountOfAddedUsers.ListAddList(x.Value(Name_UsersAdded).StringToList(Of Integer)("|"), lc)
                         CountOfLoadedPostsPerSession.ListAddList(x.Value(Name_PostsDownloaded).StringToList(Of Integer)("|"), lc)
                         ChannelExistentUserNames.ListAddList(x.Value(Name_UsersExistent).StringToList(Of String)("|"), LNC)
                         If Not PartialLoad Then
                             With x(Name_PostsNode).XmlIfNothing
                                 If .Count > 0 Then .ForEach(Sub(ee) PostsLatest.Add(New UserPost With {
                                                                         .ID = ee.Attribute(Name_ID),
-                                                                        .[Date] = AConvert(Of Date)(ee.Attribute(Name_Date).Value, XMLDateProvider, Nothing)}))
+                                                                        .[Date] = AConvert(Of Date)(ee.Attribute(Name_Date).Value, DateTimeDefaultProvider, Nothing)}))
                             End With
                         End If
                     End If
                 End Using
             End If
             Return True
         End Function
+        Private _Saved As Boolean = True
+        Friend Function SaveUnsaved() As Boolean
+            Return _Saved OrElse Save()
+        End Function
         Friend Overloads Function Save(Optional ByVal f As SFile = Nothing, Optional ByVal e As ErrorsDescriber = Nothing) As Boolean Implements ILoaderSaver.Save
-            Dim XMLDateProvider As New ADateTime(ADateTime.Formats.BaseDateTime)
-            UpdateUsersStats()
-            If Not ViewMode = View.New Then
-                Dim l As New List(Of String)
-                If Posts.Count > 0 Or PostsLatest.Count > 0 Then l.ListAddList((From p In PostsAll Where Not p.ID.IsEmptyString Select p.ID), LNC)
-                l.ListAddList(PostsNames, LNC)
-                If l.Count > 0 Then TextSaver.SaveTextToFile(l.ListToString("|"), FilePosts, True,, EDP.SendToLog)
-            End If
-            Using x As New XmlFile With {.AllowSameNames = True, .Name = "Channel"}
-                x.Add(Name_Name, Name)
-                x.Add(Name_ID, ID)
-                x.Add(Name_ViewMode, CInt(ViewMode))
-                x.Add(Name_ViewPeriod, CInt(ViewPeriod))
-                x.Add(Name_UsersAdded, CountOfAddedUsers.ListToString("|"))
-                x.Add(Name_PostsDownloaded, CountOfLoadedPostsPerSession.ListToString("|"))
-                x.Add(Name_UsersExistent, ChannelExistentUserNames.ListToString("|"))
-                If Posts.Count > 0 Or PostsLatest.Count > 0 Then
-                    Dim tmpPostList As List(Of UserPost) = Nothing
-                    tmpPostList.ListAddList(Posts).ListAddList(PostsLatest)
-                    tmpPostList.Sort()
-                    LatestParsedDate = tmpPostList.FirstOrDefault(Function(pd) pd.Date.HasValue).Date
-                    x.Add(Name_Date, AConvert(Of String)(LatestParsedDate, XMLDateProvider, String.Empty))
-                    x.Add(Name_PostsNode, String.Empty)
-                    With x(Name_PostsNode)
-                        tmpPostList.Take(200).ToList.ForEach(Sub(p) .Add(New EContainer("Post",
-                                                                                  String.Empty,
-                                                                                  {
-                                                                                    New EAttribute(Name_ID, p.ID),
-                                                                                    New EAttribute(Name_Date, AConvert(Of String)(p.Date, XMLDateProvider, String.Empty))
-                                                                                  })
-                                                                  )
-                                                      )
-                    End With
-                    tmpPostList.Clear()
+            Try
+                UpdateUsersStats()
+                If Not ViewMode = View.New Then
+                    Dim l As New List(Of String)
+                    If Posts.Count > 0 Or PostsLatest.Count > 0 Then l.ListAddList((From p In PostsAll Where Not p.ID.IsEmptyString Select p.ID), LNC)
+                    l.ListAddList(PostsNames, LNC)
+                    If l.Count > 0 Then TextSaver.SaveTextToFile(l.ListToString("|"), FilePosts, True,, EDP.SendToLog)
                 End If
-                x.Save(File)
-            End Using
-            Return True
+                Using x As New XmlFile With {.AllowSameNames = True, .Name = "Channel"}
+                    x.Add(Name_Name, Name)
+                    x.Add(Name_ID, ID)
+                    x.Add(Name_ViewMode, CInt(ViewMode))
+                    x.Add(Name_ViewPeriod, CInt(ViewPeriod))
+                    x.Add(Name_UsersAdded, CountOfAddedUsers.ListToString("|"))
+                    x.Add(Name_PostsDownloaded, CountOfLoadedPostsPerSession.ListToString("|"))
+                    x.Add(Name_UsersExistent, ChannelExistentUserNames.ListToString("|"))
+                    If Posts.Count > 0 Or PostsLatest.Count > 0 Then
+                        Dim tmpPostList As List(Of UserPost) = Nothing
+                        tmpPostList.ListAddList(Posts).ListAddList(PostsLatest)
+                        tmpPostList.Sort()
+                        LatestParsedDate = tmpPostList.FirstOrDefault(Function(pd) pd.Date.HasValue).Date
+                        x.Add(Name_Date, AConvert(Of String)(LatestParsedDate, DateTimeDefaultProvider, String.Empty))
+                        x.Add(Name_PostsNode, String.Empty)
+                        With x(Name_PostsNode)
+                            tmpPostList.Take(200).ToList.ForEach(Sub(p) .Add(New EContainer("Post",
+                                                                                      String.Empty,
+                                                                                      {
+                                                                                        New EAttribute(Name_ID, p.ID),
+                                                                                        New EAttribute(Name_Date, AConvert(Of String)(p.Date, DateTimeDefaultProvider, String.Empty))
+                                                                                      })
+                                                                      )
+                                                          )
+                        End With
+                        tmpPostList.Clear()
+                    End If
+                    _Saved = x.Save(File)
+                End Using
+                Return True
+            Catch ex As Exception
+                If Not e.Exists Then e = EDP.ReturnValue
+                Return ErrorsDescriber.Execute(e, ex, "API.Reddit.Channel.Save", _Saved)
+            End Try
         End Function
 #End Region
 #Region "IDisposable Support"