-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Test simulation attempted run without progress #103
Comments
Hi Heather, Part of your issue may be that t1>t2. See the manual part 2.5. |
Thanks @jbucholz! I fixed this but am still getting an error. t1>t2 |
I would try removing your ghost populations and see if it runs. I was having issues with those as well. It doesn't look like they are getting sampled. |
Hi, I am currently working on the next release, that should be available in the next few days. I did a lot of internal modifications, hopefully it will solve your problem. I'll keep you updated. |
@heather340 I guess it's not a true "ghost" population bc it exists in your file, but the extra N1s in the first line of your model might be throwing things off? just a thought. Thanks @gdurif for all your hard work on this! |
Thanks, and sorry I am kind of two or three weeks late on my schedule for this release but it should be worth it. |
Hi, I am really sorry, the next release was delayed to the end of June. I hope this will be ok. All apologies for the trouble. |
Hi @gdurif - that's fine! However, I'm finishing my dissertation this summer and this analysis is the very last thing that I'm waiting to add to it. Would you have any time to take a look at my prior issue in light of the current version? I was hoping to have at least a first run completed by the end of the month! |
@heather340 I will be away from office for a week but I will try to look into it when I get back. |
Perfect! Next week works better for me as well. Thank you!!
… On Jun 14, 2021, at 11:35 AM, gdurif ***@***.***> wrote:
@heather340 <https://github.com/heather340> I will be away from office for a week but I will try to look into it when I get back.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#103 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKKYHXEQOGECUG7J3T3TA6DTSYOVDANCNFSM46A5DSFQ>.
|
Hi all, Is there any update about this issue? My dataset is SNPs, 6 populations and 1907 snps. I have a similar problem, in my case after a minute since start the run, came up with "issues in run" (GUI platform). In R came up this error: I'm new in DIYABC but after different attempts (reduce my datasets, different scenarios) using the GUI platform, the error came up. I'm not sure if is something with my computer (macOS Big Sur version 11.4) or something with my input file (I used the vcf2diyabc.py script to convert my vcf). Looking forward any suggestion/update |
Hi @Belenarias - I was just going to check in today so you beat me to it. No, no progress so far. It looks like the new version is not quite out yet, but I really hope that it will solve both of our issues! |
Hi @heather340, I gave a new try this morning using different MAF values and the problem continue, no simulations. |
Hi @Belenarias @heather340 |
Hi @Belenarias I struggled with this at first but I was able to get DIYABC up and running for all my analyses. I will say if your populations are not in order in your VCF that you want for your scenarios, your scenarios will fail (for example I had 3 sampling locations in my file that were organized by site, but those sites were in alphabetical order in my VCF, not all lumped together). Once I realized this, and practiced with the toy data for setting conditions, I was able to run all my models pretty seamlessly. |
Thanks @jbucholz for the tip! I do feel like this may have been part of the issue and fixed this. However, I still haven't been able to get it going, but it does run for a few more seconds. I'm open to more suggestions if you see anything that sticks out to you in the below info: R console: End of the log:
|
@jbucholz @Belenarias @gdurif I think we are making progress! After reorganizing my file and removing my population 5, I was able to get one of my scenarios successfully simulated. However, I'm still having issues with the other scenarios and keep getting error messages. This one worked: and these two did not work: **Scenario 3 [BAD] verification de l'integrite de la table de reference dans rt.readheader this->filerefscen = /var/folders/zk/npmr9sn5449_3cd0mccfq45m0000gn/T//RtmpvJBjuq/diyabc_rf1233d2b997712/reftabscen.txt Do you have any ideas on why they might not work? Does it matter in which order I put the merges and splits? |
Or perhaps I am doing something else incorrectly: Adding more than one scenario resulted in errors, but simulating each scenario independently worked fine. Maybe the new version will correct this issue ;) |
Hi all! sorry for my late reply. @jbucholz many thanks for your tip..it helped me a lot! @heather340 thanks for your updates, you give me hope to keep trying with the program. My update, I'm still stuck in the simulation runs, but I think I found the potential issue. My snps vcf file was converted with the python script, using MAF=Hudson (also try with 0.05 and 0.01, remaining the problem)once I tried to run the simulation the "debut" come up with an unusual results: from 1179 loci, 728 from them were monomorphic loci and in somehow the program needs to rewrite the binary file (details below): purge of 728 monomorphic loci @heather340 @jbucholz @gdurif have you got these monomorphic loci in your simulation runs? Do you think it might be the issue in my simulations? I'm confusing about this point, all my data is autosome (A) and it seems the converted script reject polymorphism when is not biallelic (some of my loci are 0/0 but I found many others 0/1. Is there any other way to convert vcf to diyabc? (script used is here: https://github.com/loire/vcf2DIYABC.py) Just in case, I selected a subset from my snps (20 loci) using MAF=Hudson and again some of them came up us monomorphic (8) but at least I was able to run 600 simulation with one scenario (1000 simulations were set initially). No idea what is going on with my input file so any help/template/example will be very useful! |
Could you paste here the content of the file Regarding the conversion from VCF to DIYABC format, I do not know, I will try to get some information and get back to you. |
@Belenarias you are also motivating me! Sorry you're having trouble with monomorphic loci now. Thanks @gdurif for being willing to take a look: below I have my headerRF.txt. I included the first three scenarios that I will be testing. I have also attached my SNP file (as .txt)
|
Hi @heather340 I've checked your snp file, why did you used sex ratio 0.5 (NM=0.5NF)? as far I understood from the manual (key files section in Page 47 and 48) a balance sex ratio will be NM=1NF, maybe that could be the problem? Not sure but you can give a try. @gdurif I think my input (converted vcf it's fine) but I need to remove the monomorphic loci before submitting in the program. this is suggested in the manual Page 48 paragraph "Following Hudson's (2002) criterion". Do you have any script/idea how to removed the monomorphic snps? I'm thinking give a try in PGDspider but maybe you have more experience. Maybe due to the large amount of monomorphic in my dataset the DIYABC simulation can't run. Another alternative that might affect my simulations is the Scenarios, below you can find the "headerRF.txt" for one scenario that the simulation failed. snps_med_poporder_renamed.snp 1 scenarios: 12 historical parameters priors (9,2) loci description (1) group summary statistics (572) scenario N1 N2 N3 N4 N5 N6 ta ts tz ML1p_1 ML1p_2 ML1p_3 ML1p_4 ML1p_5 ML1p_6 ML2p_1.2 ML2p_1.3 ML2p_1.4 ML2p_1.5 ML2p_1.6 ML2p_2.3 ML2p_2.4 ML2p_2.5 ML2p_2.6 ML2p_3.4 ML2p_3.5 ML2p_3.6 ML2p_4.5 ML2p_4.6 ML2p_5.6 ML3p_1.2.3 ML3p_1.2.4 ML3p_1.2.5 ML3p_1.2.6 ML3p_1.3.4 ML3p_1.3.5 ML3p_1.3.6 ML3p_1.4.5 ML3p_1.4.6 ML3p_1.5.6 ML3p_2.3.4 ML3p_2.3.5 ML3p_2.3.6 ML3p_2.4.5 ML3p_2.4.6 ML3p_2.5.6 ML3p_3.4.5 ML3p_3.4.6 ML3p_3.5.6 ML3p_4.5.6 ML4p_1.2.3.4 ML4p_1.2.3.5 ML4p_1.2.3.6 ML4p_1.2.4.5 ML4p_1.2.4.6 ML4p_1.2.5.6 ML4p_1.3.4.5 ML4p_1.3.4.6 ML4p_1.3.5.6 ML4p_1.4.5.6 ML4p_2.3.4.5 ML4p_2.3.4.6 ML4p_2.3.5.6 ML4p_2.4.5.6 ML4p_3.4.5.6 HWm_1 HWm_2 HWm_3 HWm_4 HWm_5 HWm_6 HWv_1 HWv_2 HWv_3 HWv_4 HWv_5 HWv_6 HBm_1.2 HBm_1.3 HBm_1.4 HBm_1.5 HBm_1.6 HBm_2.3 HBm_2.4 HBm_2.5 HBm_2.6 HBm_3.4 HBm_3.5 HBm_3.6 HBm_4.5 HBm_4.6 HBm_5.6 HBv_1.2 HBv_1.3 HBv_1.4 HBv_1.5 HBv_1.6 HBv_2.3 HBv_2.4 HBv_2.5 HBv_2.6 HBv_3.4 HBv_3.5 HBv_3.6 HBv_4.5 HBv_4.6 HBv_5.6 FST1m_1 FST1m_2 FST1m_3 FST1m_4 FST1m_5 FST1m_6 FST1v_1 FST1v_2 FST1v_3 FST1v_4 FST1v_5 FST1v_6 FST2m_1.2 FST2m_1.3 FST2m_1.4 FST2m_1.5 FST2m_1.6 FST2m_2.3 FST2m_2.4 FST2m_2.5 FST2m_2.6 FST2m_3.4 FST2m_3.5 FST2m_3.6 FST2m_4.5 FST2m_4.6 FST2m_5.6 FST2v_1.2 FST2v_1.3 FST2v_1.4 FST2v_1.5 FST2v_1.6 FST2v_2.3 FST2v_2.4 FST2v_2.5 FST2v_2.6 FST2v_3.4 FST2v_3.5 FST2v_3.6 FST2v_4.5 FST2v_4.6 FST2v_5.6 NEIm_1.2 NEIm_1.3 NEIm_1.4 NEIm_1.5 NEIm_1.6 NEIm_2.3 NEIm_2.4 NEIm_2.5 NEIm_2.6 NEIm_3.4 NEIm_3.5 NEIm_3.6 NEIm_4.5 NEIm_4.6 NEIm_5.6 NEIv_1.2 NEIv_1.3 NEIv_1.4 NEIv_1.5 NEIv_1.6 NEIv_2.3 NEIv_2.4 NEIv_2.5 NEIv_2.6 NEIv_3.4 NEIv_3.5 NEIv_3.6 NEIv_4.5 NEIv_4.6 NEIv_5.6 AMLm_1.2.3 AMLm_2.1.3 AMLm_3.1.2 AMLm_1.2.4 AMLm_2.1.4 AMLm_4.1.2 AMLm_1.2.5 AMLm_2.1.5 AMLm_5.1.2 AMLm_1.2.6 AMLm_2.1.6 AMLm_6.1.2 AMLm_1.3.4 AMLm_3.1.4 AMLm_4.1.3 AMLm_1.3.5 AMLm_3.1.5 AMLm_5.1.3 AMLm_1.3.6 AMLm_3.1.6 AMLm_6.1.3 AMLm_1.4.5 AMLm_4.1.5 AMLm_5.1.4 AMLm_1.4.6 AMLm_4.1.6 AMLm_6.1.4 AMLm_1.5.6 AMLm_5.1.6 AMLm_6.1.5 AMLm_2.3.4 AMLm_3.2.4 AMLm_4.2.3 AMLm_2.3.5 AMLm_3.2.5 AMLm_5.2.3 AMLm_2.3.6 AMLm_3.2.6 AMLm_6.2.3 AMLm_2.4.5 AMLm_4.2.5 AMLm_5.2.4 AMLm_2.4.6 AMLm_4.2.6 AMLm_6.2.4 AMLm_2.5.6 AMLm_5.2.6 AMLm_6.2.5 AMLm_3.4.5 AMLm_4.3.5 AMLm_5.3.4 AMLm_3.4.6 AMLm_4.3.6 AMLm_6.3.4 AMLm_3.5.6 AMLm_5.3.6 AMLm_6.3.5 AMLm_4.5.6 AMLm_5.4.6 AMLm_6.4.5 AMLv_1.2.3 AMLv_2.1.3 AMLv_3.1.2 AMLv_1.2.4 AMLv_2.1.4 AMLv_4.1.2 AMLv_1.2.5 AMLv_2.1.5 AMLv_5.1.2 AMLv_1.2.6 AMLv_2.1.6 AMLv_6.1.2 AMLv_1.3.4 AMLv_3.1.4 AMLv_4.1.3 AMLv_1.3.5 AMLv_3.1.5 AMLv_5.1.3 AMLv_1.3.6 AMLv_3.1.6 AMLv_6.1.3 AMLv_1.4.5 AMLv_4.1.5 AMLv_5.1.4 AMLv_1.4.6 AMLv_4.1.6 AMLv_6.1.4 AMLv_1.5.6 AMLv_5.1.6 AMLv_6.1.5 AMLv_2.3.4 AMLv_3.2.4 AMLv_4.2.3 AMLv_2.3.5 AMLv_3.2.5 AMLv_5.2.3 AMLv_2.3.6 AMLv_3.2.6 AMLv_6.2.3 AMLv_2.4.5 AMLv_4.2.5 AMLv_5.2.4 AMLv_2.4.6 AMLv_4.2.6 AMLv_6.2.4 AMLv_2.5.6 AMLv_5.2.6 AMLv_6.2.5 AMLv_3.4.5 AMLv_4.3.5 AMLv_5.3.4 AMLv_3.4.6 AMLv_4.3.6 AMLv_6.3.4 AMLv_3.5.6 AMLv_5.3.6 AMLv_6.3.5 AMLv_4.5.6 AMLv_5.4.6 AMLv_6.4.5 FST3m_1.2.3 FST3m_1.2.4 FST3m_1.2.5 FST3m_1.2.6 FST3m_1.3.4 FST3m_1.3.5 FST3m_1.3.6 FST3m_1.4.5 FST3m_1.4.6 FST3m_1.5.6 FST3m_2.3.4 FST3m_2.3.5 FST3m_2.3.6 FST3m_2.4.5 FST3m_2.4.6 FST3m_2.5.6 FST3m_3.4.5 FST3m_3.4.6 FST3m_3.5.6 FST3m_4.5.6 FST3v_1.2.3 FST3v_1.2.4 FST3v_1.2.5 FST3v_1.2.6 FST3v_1.3.4 FST3v_1.3.5 FST3v_1.3.6 FST3v_1.4.5 FST3v_1.4.6 FST3v_1.5.6 FST3v_2.3.4 FST3v_2.3.5 FST3v_2.3.6 FST3v_2.4.5 FST3v_2.4.6 FST3v_2.5.6 FST3v_3.4.5 FST3v_3.4.6 FST3v_3.5.6 FST3v_4.5.6 FST4m_1.2.3.4 FST4m_1.2.3.5 FST4m_1.2.3.6 FST4m_1.2.4.5 FST4m_1.2.4.6 FST4m_1.2.5.6 FST4m_1.3.4.5 FST4m_1.3.4.6 FST4m_1.3.5.6 FST4m_1.4.5.6 FST4m_2.3.4.5 FST4m_2.3.4.6 FST4m_2.3.5.6 FST4m_2.4.5.6 FST4m_3.4.5.6 FST4v_1.2.3.4 FST4v_1.2.3.5 FST4v_1.2.3.6 FST4v_1.2.4.5 FST4v_1.2.4.6 FST4v_1.2.5.6 FST4v_1.3.4.5 FST4v_1.3.4.6 FST4v_1.3.5.6 FST4v_1.4.5.6 FST4v_2.3.4.5 FST4v_2.3.4.6 FST4v_2.3.5.6 FST4v_2.4.5.6 FST4v_3.4.5.6 FSTGm_0 FSTGv_0 F3m_1.2.3 F3m_2.1.3 F3m_3.1.2 F3m_1.2.4 F3m_2.1.4 F3m_4.1.2 F3m_1.2.5 F3m_2.1.5 F3m_5.1.2 F3m_1.2.6 F3m_2.1.6 F3m_6.1.2 F3m_1.3.4 F3m_3.1.4 F3m_4.1.3 F3m_1.3.5 F3m_3.1.5 F3m_5.1.3 F3m_1.3.6 F3m_3.1.6 F3m_6.1.3 F3m_1.4.5 F3m_4.1.5 F3m_5.1.4 F3m_1.4.6 F3m_4.1.6 F3m_6.1.4 F3m_1.5.6 F3m_5.1.6 F3m_6.1.5 F3m_2.3.4 F3m_3.2.4 F3m_4.2.3 F3m_2.3.5 F3m_3.2.5 F3m_5.2.3 F3m_2.3.6 F3m_3.2.6 F3m_6.2.3 F3m_2.4.5 F3m_4.2.5 F3m_5.2.4 F3m_2.4.6 F3m_4.2.6 F3m_6.2.4 F3m_2.5.6 F3m_5.2.6 F3m_6.2.5 F3m_3.4.5 F3m_4.3.5 F3m_5.3.4 F3m_3.4.6 F3m_4.3.6 F3m_6.3.4 F3m_3.5.6 F3m_5.3.6 F3m_6.3.5 F3m_4.5.6 F3m_5.4.6 F3m_6.4.5 F3v_1.2.3 F3v_2.1.3 F3v_3.1.2 F3v_1.2.4 F3v_2.1.4 F3v_4.1.2 F3v_1.2.5 F3v_2.1.5 F3v_5.1.2 F3v_1.2.6 F3v_2.1.6 F3v_6.1.2 F3v_1.3.4 F3v_3.1.4 F3v_4.1.3 F3v_1.3.5 F3v_3.1.5 F3v_5.1.3 F3v_1.3.6 F3v_3.1.6 F3v_6.1.3 F3v_1.4.5 F3v_4.1.5 F3v_5.1.4 F3v_1.4.6 F3v_4.1.6 F3v_6.1.4 F3v_1.5.6 F3v_5.1.6 F3v_6.1.5 F3v_2.3.4 F3v_3.2.4 F3v_4.2.3 F3v_2.3.5 F3v_3.2.5 F3v_5.2.3 F3v_2.3.6 F3v_3.2.6 F3v_6.2.3 F3v_2.4.5 F3v_4.2.5 F3v_5.2.4 F3v_2.4.6 F3v_4.2.6 F3v_6.2.4 F3v_2.5.6 F3v_5.2.6 F3v_6.2.5 F3v_3.4.5 F3v_4.3.5 F3v_5.3.4 F3v_3.4.6 F3v_4.3.6 F3v_6.3.4 F3v_3.5.6 F3v_5.3.6 F3v_6.3.5 F3v_4.5.6 F3v_5.4.6 F3v_6.4.5 F4m_1.2.3.4 F4m_1.3.2.4 F4m_1.4.2.3 F4m_1.2.3.5 F4m_1.3.2.5 F4m_1.5.2.3 F4m_1.2.3.6 F4m_1.3.2.6 F4m_1.6.2.3 F4m_1.2.4.5 F4m_1.4.2.5 F4m_1.5.2.4 F4m_1.2.4.6 F4m_1.4.2.6 F4m_1.6.2.4 F4m_1.2.5.6 F4m_1.5.2.6 F4m_1.6.2.5 F4m_1.3.4.5 F4m_1.4.3.5 F4m_1.5.3.4 F4m_1.3.4.6 F4m_1.4.3.6 F4m_1.6.3.4 F4m_1.3.5.6 F4m_1.5.3.6 F4m_1.6.3.5 F4m_1.4.5.6 F4m_1.5.4.6 F4m_1.6.4.5 F4m_2.3.4.5 F4m_2.4.3.5 F4m_2.5.3.4 F4m_2.3.4.6 F4m_2.4.3.6 F4m_2.6.3.4 F4m_2.3.5.6 F4m_2.5.3.6 F4m_2.6.3.5 F4m_2.4.5.6 F4m_2.5.4.6 F4m_2.6.4.5 F4m_3.4.5.6 F4m_3.5.4.6 F4m_3.6.4.5 F4v_1.2.3.4 F4v_1.3.2.4 F4v_1.4.2.3 F4v_1.2.3.5 F4v_1.3.2.5 F4v_1.5.2.3 F4v_1.2.3.6 F4v_1.3.2.6 F4v_1.6.2.3 F4v_1.2.4.5 F4v_1.4.2.5 F4v_1.5.2.4 F4v_1.2.4.6 F4v_1.4.2.6 F4v_1.6.2.4 F4v_1.2.5.6 F4v_1.5.2.6 F4v_1.6.2.5 F4v_1.3.4.5 F4v_1.4.3.5 F4v_1.5.3.4 F4v_1.3.4.6 F4v_1.4.3.6 F4v_1.6.3.4 F4v_1.3.5.6 F4v_1.5.3.6 F4v_1.6.3.5 F4v_1.4.5.6 F4v_1.5.4.6 F4v_1.6.4.5 F4v_2.3.4.5 F4v_2.4.3.5 F4v_2.5.3.4 F4v_2.3.4.6 F4v_2.4.3.6 F4v_2.6.3.4 F4v_2.3.5.6 F4v_2.5.3.6 F4v_2.6.3.5 F4v_2.4.5.6 F4v_2.5.4.6 F4v_2.6.4.5 F4v_3.4.5.6 F4v_3.5.4.6 F4v_3.6.4.5 |
@Belenarias I see, there is at least one issue with the generated The line In the mean time, I tried to use the CLI tools (see https://diyabc.github.io/cli/) to run your example with your dataset but I am still encountering an issue with the |
@Belenarias thanks for noticing that - I had just put in the default as a mistake. But, it doesn't seem to have affected the issue. |
@gdurif Many thanks for letting me know about the bug so quick!. I hope my dataset helps to improve the program. When do you think will be available the new release? I do need to run this analysis for a resubmission so my time is tight. Thanks for your effort and support. |
Hi @ALL, I am again really sorry for the delay, but the new version of the GUI is at least available on branch If you want to try it first hand, you can install the R package and run the app with: devtools::install_github("diyabc/diyabcGUI", ref = "refactor_app", subdir = "R-pkg")
library(diyabcGUI)
dl_all_latest_bin()
diyabc() It will be merged tomorrow in Edit (2021/08/24): I am encountering a network issue delaying the release build (always the final step...), hopefully it will be fixed in a few hours. |
The new release is available. |
Hi @heather340 |
So sorry to hear that it's giving you issues! Good luck with the fix. |
Hi, I finally found the problem, new release build is now in progress, see issue #116 for a summary about this matter. |
The release is out: v1.1.1-beta |
Fixed by d412012 |
Hi! I'm running SNP data and am having problems getting it to run. I left it running for ~5 hrs yesterday and no progress was made in the run log so I stopped it. Now I keep getting an error once I restarted the GUI but the error is not defined within the run log so I'm not sure where the issue lies. Would it be possible to send my data and a model to someone to see if another set of eyes can determine where I'm making a mistake?
I used the vcf to DIYABC script to convert my file over, and made sure I had spaces instead of tabs within my SNP data file.
Data file info
Sex ratio: NM=0.5NF
Minimum allele frequency criterion: MAF=hudson
Additional information: DIYABC_SNP_800_75
header: 'IND SEX POP A A A A A A A A A A A A A A A A A ...'
Sample: 63 individuals from 5 populations
Total number of loci = 1065
Available loci: 893 (note: 172 loci are filtered out based on MAF criterion)
Sample historical model:
N1 N2 N3 N4 N5 N1 N1 N1 N1
0 sample 1
0 sample 2
0 sample 3
0 sample 4
0 sample 5
t1 merge 3 5
t2 merge 2 4
t3 merge 2 3
t4 merge 2 1
Priors and conditions:
Keep as Uniform
N1 -> N5:
min: 1000
max: 1000000
(min-max)
t1
100-100000
t2
1000-1000000
t3
10000-1000000
t4
10000-1000000
Condition setting:
t1<t2
t2<t3
t1<t3
The text was updated successfully, but these errors were encountered: