Added API tests for file upload and file download

[nikolay-advolodkin]

This is a new convention that we don't really have in any of our automation where we are mixing API and UI tests using Cucumber. We are always preaching to our clients to do this, but we don't have an example. I created such examples for commonly asked scenarios of testing file upload and download. These suites may not be complete, but it should give clients a good idea into how to proceed forward.

Please take a look at the new API tests I added and if you like my implementation

[sauceaaron]

I have a number of comments that don't really fit in a code review. 1. While this type of example is useful, I don't think it belongs in our sample test frameworks. My impression is that these are designed to help Sauce users understand how to use Sauce Labs (in the context of a Selenium or Appium test) and should be kept as simple as possible while still demonstrating best practices. I think there is a place for additional test code that shows how to use REST APIs -- especially to demonstrate how using REST APIs can integrate with Selenium tests to make Selenium do less. With file download in particular, I think an example of how to do so would go well on the wiki page that says "How do I download files to use Selenium? --- Answer: You don't. And here's why... BTW, here is sample code that shows you how you can download a file using RestAssured". 2. I don't love RestAssured. Because it depends on static methods to make it more readable, it's not great for parallelization. It can be done without, but it assumes that you do use the static responses/etc. And also it's difficult and downright ugly if you get out of it's default given(baseUrl).with(params).when().get(path).then().statusCode(200).... etc It actually makes it harder to get the data you want to test than other frameworks because it really only is targeting simple use cases in a readable style. 3. RestAssured doesn't play well with Cucumber. It has it's own given().when().then() syntax but unfortunately, doesn't map well to Cucumber's @Given/@When/@then blocks because of it's use of static state and because Cucumber can't pass state between Step Definition files without using a Dependency Injection framework. 4. File download is not really a REST API action. Really, you just want to make a simple HTTP request. It's true that a REST framework might make it easier, but RestAssured is not only a REST client, but it's own testing ecosystem. It makes it harder to just get a file -- or in this case, probably all you really want to do is check that the file is downloadabout using a HEAD request, not a GET request -- unless you actually want to parse the file contents. And that should probably be done outside of the whole HTTP request cycle anyway. Better to have a "unit" (integration) test query the file generator locally and then parse it with whatever file parsing tool you're going to use (awk or iText or poi or something) You can see how a simple file download doesn't really fit into a cucumber scenario by how much meaningless boilerplate you're creating, and how much of cucumber you're actually ignoring. I realize some of our customers use Cucumber because it has been mandated, but we can do better. It actually might make sense in the context of a Selenium test to use a REST API (or download a file with a HTTP client) to avoid Selenium steps and make the tests more reliable or complete. Such as data setup or file download. But I don't think it fits into our sample framework, and trying to demonstrate with Cucumber seems like folly. At worst, someone using cucumber should be able to take an example code snippet (3-5 lines of code) that downloads a file from another example and place that within a single step in a larger scenario. Given I have paid for my book When I go to the download page And I download the PDF version of the book Then I should be able to read the PDF file @given("I have paid for my book") public void pay_for_book() { HttpResponse<JsonNode> response = bookstoreAPI.login(username, password) .addItem(book) .purchase(paymentMethod) .send(); JsonNode json = response.getBody(); authToken = json.getObject().get("authToken").toString(); } @when("I go to download page") public void go_to_download_page() { driver.manage().addCookie(new Cookie("authToken", authToken)); driver.get(downloadURL); downloadURL = driver.findElement(By.id("downloadPDFLink")).getAttribute("href"); } @when("I download the PDF version of the book") public void download_book() throws Exception { response = RestAssured.head(downloadURL)); if (response.statusCode() != 200) { throw new Exception("Book did not download"); } } @then("I should be able to read the FDP File") public void readPDF() { assertThat(response.getContentType(), is(equalTo("application/pdf"))); }

…

On Tue, Mar 12, 2019 at 2:05 PM Nikolay ***@***.***> wrote: @nikolay-advolodkin <https://github.com/nikolay-advolodkin> requested your review on: #13 <#13> Added API tests for file upload and file download. — You are receiving this because your review was requested. Reply to this email directly, view it on GitHub <#13 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AWZUEcRCwmL5F4LKoth0zH0pk8ElntVpks5vWAiQgaJpZM4brxU1> .

-- *Aaron Evans* Test Automation Advisor | Sauce Labs 116 New Montgomery St #300 San Francisco, CA 94105 855-677-0011 Sauce Labs <http://saucelabs.com/> | Sauce Blog <http://sauceio.com/> <https://saucelabs.com/>

[fijiaaron]

I would use RestAssured's given().baseUri(url) and when().get(path) with Cucumber

[fijiaaron]

Most of the time, you don't want to download the file -- just check that it can be downloaded:

Instead of get(url).statusCode(200);
you can do head(url).statusCode(200); and save bandwidth.

Unless, of course, you're inspecting the file, as you're doing here. But do you really want to inspect the file, or is this just a contrived example?

[fijiaaron]

Why separate API/Not API. I think the most common use case would be to use the API in conjunction with Selenium -- you sign up, navigate to the download page, and then get the download URL in Selenium, then use the HTTP client to do the actual download.

[nadvolod]

Because if I don't do this then a VM session is created for every single one of my API tests. I don't need a VM for my API test.

[mmerrell-sauce]

File download is not really a REST API action

This really clicked for me--we should be using more standard HTTP library stuff for performing what amounts to an API operation, rather than using RestAssured, which is a testing engine. We're not testing the API at this point, we're performing an operation.

I understand, @nikolay-advolodkin , if you don't know how to do that in Java... it's... painful. But I agree with the overall point, that we probably shouldn't do it here.

@fijiaaron do you think this should be in a Gist, linked from here? or from the Sauce wiki, like you indicated?

[sauceaaron]

Here's a gist of my (slightly) cleaned up (pseudo) code https://gist.github.com/fijiaaron/8eb62e0eb2647cc2133e9dd3899c69f8 I do think it would be nice to have sample code for checking a file for download. I think there is (or was) a wiki page that essentially pointed to this definitive blog post http://ardesco.lazerycode.com/testing/webdriver/2012/07/25/how-to-download-files-with-selenium-and-why-you-shouldnt.html Including a link to an actual working gist would be nice, but I say we let Dave Haeffner maintain it ;) http://elementalselenium.com/tips/8-download-a-file-revisited

…

On Tue, Mar 12, 2019 at 4:22 PM M Merrell ***@***.***> wrote: File download is not really a REST API action This really clicked for me--we should be using more standard HTTP library stuff for performing what amounts to an API operation, rather than using RestAssured, which is a testing engine. We're not testing the API at this point, we're performing an operation. I understand, Nikolay, if you don't know how to do that in Java... it's... painful. But I agree with the overall point, that we probably shouldn't do it here. @fijiaaron <https://github.com/fijiaaron> do you think this should be in a Gist, linked from here? or from the Sauce wiki, like you indicated? — You are receiving this because your review was requested. Reply to this email directly, view it on GitHub <#13 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AWZUEYzRZGnpevBmfZaF-FQkdKHUY1AIks5vWCi6gaJpZM4brxU1> .

-- *Aaron Evans* Test Automation Advisor | Sauce Labs 116 New Montgomery St #300 San Francisco, CA 94105 855-677-0011 Sauce Labs <http://saucelabs.com/> | Sauce Blog <http://sauceio.com/> <https://saucelabs.com/>

[sauceaaron]

But I also think there is room for demonstrating how to include an actual API call. Does the saucedemo have an API? On Tue, Mar 12, 2019 at 4:56 PM Aaron Evans <[email protected]> wrote:

…

Here's a gist of my (slightly) cleaned up (pseudo) code https://gist.github.com/fijiaaron/8eb62e0eb2647cc2133e9dd3899c69f8 I do think it would be nice to have sample code for checking a file for download. I think there is (or was) a wiki page that essentially pointed to this definitive blog post http://ardesco.lazerycode.com/testing/webdriver/2012/07/25/how-to-download-files-with-selenium-and-why-you-shouldnt.html Including a link to an actual working gist would be nice, but I say we let Dave Haeffner maintain it ;) http://elementalselenium.com/tips/8-download-a-file-revisited On Tue, Mar 12, 2019 at 4:22 PM M Merrell ***@***.***> wrote: > File download is not really a REST API action > > This really clicked for me--we should be using more standard HTTP library > stuff for performing what amounts to an API operation, rather than using > RestAssured, which is a testing engine. We're not testing the API at this > point, we're performing an operation. > > I understand, Nikolay, if you don't know how to do that in Java... > it's... painful. But I agree with the overall point, that we probably > shouldn't do it here. > > @fijiaaron <https://github.com/fijiaaron> do you think this should be in > a Gist, linked from here? or from the Sauce wiki, like you indicated? > > — > You are receiving this because your review was requested. > Reply to this email directly, view it on GitHub > <#13 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AWZUEYzRZGnpevBmfZaF-FQkdKHUY1AIks5vWCi6gaJpZM4brxU1> > . > -- *Aaron Evans* Test Automation Advisor | Sauce Labs 116 New Montgomery St #300 San Francisco, CA 94105 855-677-0011 Sauce Labs <http://saucelabs.com/> | Sauce Blog <http://sauceio.com/> <https://saucelabs.com/>

-- *Aaron Evans* Test Automation Advisor | Sauce Labs 116 New Montgomery St #300 San Francisco, CA 94105 855-677-0011 Sauce Labs <http://saucelabs.com/> | Sauce Blog <http://sauceio.com/> <https://saucelabs.com/>

[nikolay-advolodkin]

Thanks so much for taking the time to write that up for me. I appreciate the feedback. "My impression is that these are designed to help Sauce users understand how to use Sauce Labs (in the context of a Selenium or Appium test) " Then I believe that you and I have a fundamental difference on what a sample framework is. I believe that a sample framework should encompass all the components required to make a customer successful in test automation. As simple as possible, yes. But as I've seen time and time again, customers go in the wrong direction with our code. So the more guidance that we give them, the better. Not sure if you agree with that point? My belief is that more guidance from us = more success. And the inverse is true. In this case, a customer specifically asked me for a sample Cucumber Java framework that can run in parallel. We didn't have it, now we do and they know how to do that based on our recommendations. They also wanted guidance in how to test file stuff which is part of their test automation framework. And even though we're always preaching APIs, we have Zero code to show any customers how to do it right. Finally, I decided to integrate the API/UI because it's either I integrate it and show the customer the preferred approach, or I let them try to integrate the code into our framework and likely make some technical mess. Actually, nobody on the SA team knew how to even solve this challenge, I was the first one. Imagine the customer trying to figure out how to have a Cucumber parallel solution with API tests. 2. I don't love RestAssured. Thanks, my first time using it. Do you have a client that you recommend? 3. RestAssured doesn't Agreed, I saw that and felt a little awkard myself. I'm open to using another client. 4. Better to have a "unit" (integration) test query the file generator locally and then parse it with whatever file parsing tool you're going to use (awk or iText or poi or something) Agreed, but don't you think it's better to provide an API solution to all the people trying to use Selenium rather than telling them to ask Developers to write unit tests. The latter is right, but it's much harder, and much less likely to happen. I believe in baby steps, especially with clients. Just because a solution isn't perfect, doesn't mean that it's not better than what they currently have. "At worst, someone using cucumber should be able to take an example code snippet (3-5 lines of code) that downloads a file from another example and place that within a single step in a larger scenario." I don't think that's true. Nobody on our SA team knew how to have API tests integrated into a Cucumber solution that runs in parallel. I had to solve that problem on my own after a few hours of struggle. The integration was actually the hardest part, so imagine pushing that challenge onto our clients. We have no example of how to combine API and UI tests into a single framework and yet we're always preaching automation pyramid. This solution bridges that gap. Yes, it could be more useful with our Sample App. But does it make that big of a difference? I solved a customers problem right now with a single solution that fits their exact needs. And now, we have a working prototype that other customers can apply to their problems. It might need to be tweaked, but the parallelization, Cucumber, java, UI, API components are ready. I feel like we can continue talking about doing more API and unit testing, or we can just show them. Let me know your thoughts *Nikolay Advolodkin* Solution Architect | Sauce Labs Ft Lauderdale, FL <https://twitter.com/Nikolay_A00> <https://www.linkedin.com/in/nikolayadvolodkin/> On Tue, Mar 12, 2019 at 6:13 PM Aaron Evans <[email protected]> wrote:

…

I have a number of comments that don't really fit in a code review. 1. While this type of example is useful, I don't think it belongs in our sample test frameworks. My impression is that these are designed to help Sauce users understand how to use Sauce Labs (in the context of a Selenium or Appium test) and should be kept as simple as possible while still demonstrating best practices. I think there is a place for additional test code that shows how to use REST APIs -- especially to demonstrate how using REST APIs can integrate with Selenium tests to make Selenium do less. With file download in particular, I think an example of how to do so would go well on the wiki page that says "How do I download files to use Selenium? --- Answer: You don't. And here's why... BTW, here is sample code that shows you how you can download a file using RestAssured". 2. I don't love RestAssured. Because it depends on static methods to make it more readable, it's not great for parallelization. It can be done without, but it assumes that you do use the static responses/etc. And also it's difficult and downright ugly if you get out of it's default given(baseUrl).with(params).when().get(path).then().statusCode(200).... etc It actually makes it harder to get the data you want to test than other frameworks because it really only is targeting simple use cases in a readable style. 3. RestAssured doesn't play well with Cucumber. It has it's own given().when().then() syntax but unfortunately, doesn't map well to Cucumber's @***@***.******@***.*** blocks because of it's use of static state and because Cucumber can't pass state between Step Definition files without using a Dependency Injection framework. 4. File download is not really a REST API action. Really, you just want to make a simple HTTP request. It's true that a REST framework might make it easier, but RestAssured is not only a REST client, but it's own testing ecosystem. It makes it harder to just get a file -- or in this case, probably all you really want to do is check that the file is downloadabout using a HEAD request, not a GET request -- unless you actually want to parse the file contents. And that should probably be done outside of the whole HTTP request cycle anyway. Better to have a "unit" (integration) test query the file generator locally and then parse it with whatever file parsing tool you're going to use (awk or iText or poi or something) You can see how a simple file download doesn't really fit into a cucumber scenario by how much meaningless boilerplate you're creating, and how much of cucumber you're actually ignoring. I realize some of our customers use Cucumber because it has been mandated, but we can do better. It actually might make sense in the context of a Selenium test to use a REST API (or download a file with a HTTP client) to avoid Selenium steps and make the tests more reliable or complete. Such as data setup or file download. But I don't think it fits into our sample framework, and trying to demonstrate with Cucumber seems like folly. At worst, someone using cucumber should be able to take an example code snippet (3-5 lines of code) that downloads a file from another example and place that within a single step in a larger scenario. Given I have paid for my book When I go to the download page And I download the PDF version of the book Then I should be able to read the PDF file @given("I have paid for my book") public void pay_for_book() { HttpResponse<JsonNode> response = bookstoreAPI.login(username, password) .addItem(book) .purchase(paymentMethod) .send(); JsonNode json = response.getBody(); authToken = json.getObject().get("authToken").toString(); } @when("I go to download page") public void go_to_download_page() { driver.manage().addCookie(new Cookie("authToken", authToken)); driver.get(downloadURL); downloadURL = driver.findElement(By.id("downloadPDFLink")).getAttribute("href"); } @when("I download the PDF version of the book") public void download_book() throws Exception { response = RestAssured.head(downloadURL)); if (response.statusCode() != 200) { throw new Exception("Book did not download"); } } @then("I should be able to read the FDP File") public void readPDF() { assertThat(response.getContentType(), is(equalTo("application/pdf"))); } On Tue, Mar 12, 2019 at 2:05 PM Nikolay ***@***.***> wrote: > @nikolay-advolodkin <https://github.com/nikolay-advolodkin> requested > your review on: #13 > < #13 > > Added API tests for file upload and file download. > > — > You are receiving this because your review was requested. > Reply to this email directly, view it on GitHub > < #13 (comment) >, > or mute the thread > < https://github.com/notifications/unsubscribe-auth/AWZUEcRCwmL5F4LKoth0zH0pk8ElntVpks5vWAiQgaJpZM4brxU1 > > . > -- *Aaron Evans* Test Automation Advisor | Sauce Labs 116 New Montgomery St #300 San Francisco, CA 94105 855-677-0011 Sauce Labs <http://saucelabs.com/> | Sauce Blog <http://sauceio.com/> <https://saucelabs.com/> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#13 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AmkEcggYm0tb1WLiURsY_SVteMX3Y8CAks5vWCaegaJpZM4brxU1> .

[nikolay-advolodkin]

File download is not really a REST API action

This really clicked for me--we should be using more standard HTTP library stuff for performing what amounts to an API operation, rather than using RestAssured, which is a testing engine. We're not testing the API at this point, we're performing an operation.

I understand, @nikolay-advolodkin , if you don't know how to do that in Java... it's... painful. But I agree with the overall point, that we probably shouldn't do it here.

@fijiaaron do you think this should be in a Gist, linked from here? or from the Sauce wiki, like you indicated?

@mmerrell-sauce agreed on the REST Assured stuff. I wasn't aware of another HTTP client, so I just went with this one. I will update to use some other HTTP client 👍

However, I would still like to clarify one point on which we seem to be on a different page. Just for my understanding, are you suggesting that we should leave the integration of API and UI automation into a sample framework up to our client?

This was actually the hardest part and nobody on the SA team knew how to combine API/UI automation into a single Cucumber framework that runs in parallel. As a result, I spent a few hours to resolve this problem. If we move this example into it is own repo or Git Gist, then we are leaving the client with the responsibility of figuring out how to integrate API and UI automation into a single framework and make it work correctly.

If that's what we want to do as a team, then that's the decision. But it seems scary to me that a couple of expert SAs took hours to resolve this issue, meaning it was tough. And we will obscure that solution from that client and will just leave it on them to figure it out.

Again, fine if we want to do that. I just wanted to make sure that we all understand the consequences of separating this example out 👍

[sauceaaron]

I think we all know how to do this -- you just call webdriver with one statement: driver.get(page) and call the the API with another statement: http.get(file) And I do think we should leave this up to the client. We don't want to be responsible for writing customer code -- especially code unrelated to Sauce Labs services -- without getting paid for it and having them sign a release. That's why our sample frameworks have ALL CAPS SCARE TEXT in the readme. Once, when I did tech support for an ISP (back in the dial up modem days) I helped a user install TCP/IP on their Windows 95 computer. He then blamed me for problems with his printer and actually vandalized the office by throwing a brick through the Window. We do not want customers feeling that we are liable for testing their APIs or their file downloads. Or even assuming that it has anything to do with Sauce Labs.

…

On Mon, Mar 18, 2019 at 8:44 AM Nikolay ***@***.***> wrote: File download is not really a REST API action This really clicked for me--we should be using more standard HTTP library stuff for performing what amounts to an API operation, rather than using RestAssured, which is a testing engine. We're not testing the API at this point, we're performing an operation. I understand, @nikolay-advolodkin <https://github.com/nikolay-advolodkin> , if you don't know how to do that in Java... it's... painful. But I agree with the overall point, that we probably shouldn't do it here. @fijiaaron <https://github.com/fijiaaron> do you think this should be in a Gist, linked from here? or from the Sauce wiki, like you indicated? @mmerrell-sauce <https://github.com/mmerrell-sauce> agreed on the REST Assured stuff. I wasn't aware of another HTTP client, so I just went with this one. I will update to use some other HTTP client 👍 However, I would still like to clarify one point on which we seem to be on a different page. Just for my understanding, are you suggesting that we should leave the integration of API and UI automation into a sample framework up to our client? This was actually the hardest part and nobody on the SA team knew how to combine API/UI automation into a single Cucumber framework that runs in parallel. As a result, I spent a few hours to resolve this problem. If we move this example into it is own repo or Git Gist, then we are leaving the client with the responsibility of figuring out how to integrate API and UI automation into a single framework and make it work correctly. If that's what we want to do as a team, then that's the decision. But it seems scary to me that a couple of expert SAs took hours to resolve this issue, meaning it was tough. And we will obscure that solution from that client and will just leave it on them to figure it out. Again, fine if we want to do that. I just wanted to make sure that we all understand the consequences of separating this example out 👍 — You are receiving this because your review was requested. Reply to this email directly, view it on GitHub <#13 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AWZUEVbhEAW3z7vrBzrgBlzpDZRFJi7cks5vX6Y5gaJpZM4brxU1> .

-- *Aaron Evans* Test Automation Advisor | Sauce Labs 116 New Montgomery St #300 San Francisco, CA 94105 855-677-0011 Sauce Labs <http://saucelabs.com/> | Sauce Blog <http://sauceio.com/> <https://saucelabs.com/>

[mmerrell-sauce]

I think we should provide as much help to our clients as possible, we just need to agree on where the help should go. Should we make these frameworks "all things to all people", or should each snippet of code explain one small aspect of what we need to provide? This sounds like a discussion we should have, and I'll only hand down a decision if we can't reach consensus. I bet we will. I want to schedule time for us as a group at SauceCon, so I'll put this high on the list of things to talk about. MM On Mon, Mar 18, 2019 at 10:00 AM Aaron Evans <[email protected]> wrote:

…

I think we all know how to do this -- you just call webdriver with one statement: driver.get(page) and call the the API with another statement: http.get(file) And I do think we should leave this up to the client. We don't want to be responsible for writing customer code -- especially code unrelated to Sauce Labs services -- without getting paid for it and having them sign a release. That's why our sample frameworks have ALL CAPS SCARE TEXT in the readme. Once, when I did tech support for an ISP (back in the dial up modem days) I helped a user install TCP/IP on their Windows 95 computer. He then blamed me for problems with his printer and actually vandalized the office by throwing a brick through the Window. We do not want customers feeling that we are liable for testing their APIs or their file downloads. Or even assuming that it has anything to do with Sauce Labs. On Mon, Mar 18, 2019 at 8:44 AM Nikolay ***@***.***> wrote: > File download is not really a REST API action > > This really clicked for me--we should be using more standard HTTP library > stuff for performing what amounts to an API operation, rather than using > RestAssured, which is a testing engine. We're not testing the API at this > point, we're performing an operation. > > I understand, @nikolay-advolodkin <https://github.com/nikolay-advolodkin > > , if you don't know how to do that in Java... it's... painful. But I agree > with the overall point, that we probably shouldn't do it here. > > @fijiaaron <https://github.com/fijiaaron> do you think this should be in > a Gist, linked from here? or from the Sauce wiki, like you indicated? > > @mmerrell-sauce <https://github.com/mmerrell-sauce> agreed on the REST > Assured stuff. I wasn't aware of another HTTP client, so I just went with > this one. I will update to use some other HTTP client 👍 > > However, I would still like to clarify one point on which we seem to be on > a different page. Just for my understanding, are you suggesting that we > should leave the integration of API and UI automation into a sample > framework up to our client? > > This was actually the hardest part and nobody on the SA team knew how to > combine API/UI automation into a single Cucumber framework that runs in > parallel. As a result, I spent a few hours to resolve this problem. If we > move this example into it is own repo or Git Gist, then we are leaving the > client with the responsibility of figuring out how to integrate API and UI > automation into a single framework and make it work correctly. > > If that's what we want to do as a team, then that's the decision. But it > seems scary to me that a couple of expert SAs took hours to resolve this > issue, meaning it was tough. And we will obscure that solution from that > client and will just leave it on them to figure it out. > > Again, fine if we want to do that. I just wanted to make sure that we all > understand the consequences of separating this example out 👍 > > — > You are receiving this because your review was requested. > Reply to this email directly, view it on GitHub > < #13 (comment) >, > or mute the thread > < https://github.com/notifications/unsubscribe-auth/AWZUEVbhEAW3z7vrBzrgBlzpDZRFJi7cks5vX6Y5gaJpZM4brxU1 > > . > -- *Aaron Evans* Test Automation Advisor | Sauce Labs 116 New Montgomery St #300 San Francisco, CA 94105 855-677-0011 Sauce Labs <http://saucelabs.com/> | Sauce Blog <http://sauceio.com/> <https://saucelabs.com/> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#13 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ArrvwtO2p-0sklF2WxYDxmaGp9fp1Igsks5vX6oAgaJpZM4brxU1> .

-- *Marcus Merrell* Director of Technical Services | Sauce Labs 116 New Montgomery Street, Suite 300 San Francisco, CA 94105 (M) +1 512-619-9534 Sauce Labs <http://saucelabs.com/> | Sauce blog <http://saucelabs.com/blog>

[nikolay-advolodkin]

That's a good point that you bring up. Makes sense. Just to make sure that I fully understand your point of view, not trying to be disagreeable: - So writing a code sample to show file download is okay. - Writing a sample Cucumber framework that runs in parallel is okay. - Integrating the 2 code samples together is outside of the scope of what SAs should do? *Nikolay Advolodkin* Solution Architect | Sauce Labs Ft Lauderdale, FL <https://twitter.com/Nikolay_A00> <https://www.linkedin.com/in/nikolayadvolodkin/> On Mon, Mar 18, 2019 at 11:00 AM Aaron Evans <[email protected]> wrote:

…

I think we all know how to do this -- you just call webdriver with one statement: driver.get(page) and call the the API with another statement: http.get(file) And I do think we should leave this up to the client. We don't want to be responsible for writing customer code -- especially code unrelated to Sauce Labs services -- without getting paid for it and having them sign a release. That's why our sample frameworks have ALL CAPS SCARE TEXT in the readme. Once, when I did tech support for an ISP (back in the dial up modem days) I helped a user install TCP/IP on their Windows 95 computer. He then blamed me for problems with his printer and actually vandalized the office by throwing a brick through the Window. We do not want customers feeling that we are liable for testing their APIs or their file downloads. Or even assuming that it has anything to do with Sauce Labs. On Mon, Mar 18, 2019 at 8:44 AM Nikolay ***@***.***> wrote: > File download is not really a REST API action > > This really clicked for me--we should be using more standard HTTP library > stuff for performing what amounts to an API operation, rather than using > RestAssured, which is a testing engine. We're not testing the API at this > point, we're performing an operation. > > I understand, @nikolay-advolodkin <https://github.com/nikolay-advolodkin > > , if you don't know how to do that in Java... it's... painful. But I agree > with the overall point, that we probably shouldn't do it here. > > @fijiaaron <https://github.com/fijiaaron> do you think this should be in > a Gist, linked from here? or from the Sauce wiki, like you indicated? > > @mmerrell-sauce <https://github.com/mmerrell-sauce> agreed on the REST > Assured stuff. I wasn't aware of another HTTP client, so I just went with > this one. I will update to use some other HTTP client 👍 > > However, I would still like to clarify one point on which we seem to be on > a different page. Just for my understanding, are you suggesting that we > should leave the integration of API and UI automation into a sample > framework up to our client? > > This was actually the hardest part and nobody on the SA team knew how to > combine API/UI automation into a single Cucumber framework that runs in > parallel. As a result, I spent a few hours to resolve this problem. If we > move this example into it is own repo or Git Gist, then we are leaving the > client with the responsibility of figuring out how to integrate API and UI > automation into a single framework and make it work correctly. > > If that's what we want to do as a team, then that's the decision. But it > seems scary to me that a couple of expert SAs took hours to resolve this > issue, meaning it was tough. And we will obscure that solution from that > client and will just leave it on them to figure it out. > > Again, fine if we want to do that. I just wanted to make sure that we all > understand the consequences of separating this example out 👍 > > — > You are receiving this because your review was requested. > Reply to this email directly, view it on GitHub > < #13 (comment) >, > or mute the thread > < https://github.com/notifications/unsubscribe-auth/AWZUEVbhEAW3z7vrBzrgBlzpDZRFJi7cks5vX6Y5gaJpZM4brxU1 > > . > -- *Aaron Evans* Test Automation Advisor | Sauce Labs 116 New Montgomery St #300 San Francisco, CA 94105 855-677-0011 Sauce Labs <http://saucelabs.com/> | Sauce Blog <http://sauceio.com/> <https://saucelabs.com/> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#13 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AmkEci-wLKRJTvXhtgDONXXvFkW_G-RYks5vX6oAgaJpZM4brxU1> .

[mmerrell-sauce]

- Writing a code sample to show damn near anything that will benefit the customer is okay, as long as it's good code, and we all have confidence in its quality - I personally like the idea of integrating the 2 code samples into a single framework, and I don't think it's outside the scope of what SAs should do, but I don't believe there's general agreement among the SAs about that, and I think we need to talk about it For now, if you need to share those two concepts combined, for a customer, nothing should prevent you from doing that. I think the only point of contention here is what we provide in our github for sample frameworks. We're not going to answer it today, but we should answer it sometime in Q2. MM

…

On Mon, Mar 18, 2019 at 10:26 AM Nikolay ***@***.***> wrote: That's a good point that you bring up. Makes sense. Just to make sure that I fully understand your point of view, not trying to be disagreeable: - So writing a code sample to show file download is okay. - Writing a sample Cucumber framework that runs in parallel is okay. - Integrating the 2 code samples together is outside of the scope of what SAs should do? *Nikolay Advolodkin* Solution Architect | Sauce Labs Ft Lauderdale, FL <https://twitter.com/Nikolay_A00> <https://www.linkedin.com/in/nikolayadvolodkin/> On Mon, Mar 18, 2019 at 11:00 AM Aaron Evans ***@***.***> wrote: > I think we all know how to do this -- you just call webdriver with one > statement: > > driver.get(page) > > and call the the API with another statement: > > http.get(file) > > And I do think we should leave this up to the client. We don't want to be > responsible for writing customer code -- especially code unrelated to Sauce > Labs services -- without getting paid for it and having them sign a > release. That's why our sample frameworks have ALL CAPS SCARE TEXT in the > readme. > > Once, when I did tech support for an ISP (back in the dial up modem days) I > helped a user install TCP/IP on their Windows 95 computer. He then blamed > me for problems with his printer and actually vandalized the office by > throwing a brick through the Window. > > We do not want customers feeling that we are liable for testing their APIs > or their file downloads. Or even assuming that it has anything to do with > Sauce Labs. > > > On Mon, Mar 18, 2019 at 8:44 AM Nikolay ***@***.***> wrote: > > > File download is not really a REST API action > > > > This really clicked for me--we should be using more standard HTTP library > > stuff for performing what amounts to an API operation, rather than using > > RestAssured, which is a testing engine. We're not testing the API at this > > point, we're performing an operation. > > > > I understand, @nikolay-advolodkin < https://github.com/nikolay-advolodkin > > > > , if you don't know how to do that in Java... it's... painful. But I > agree > > with the overall point, that we probably shouldn't do it here. > > > > @fijiaaron <https://github.com/fijiaaron> do you think this should be in > > a Gist, linked from here? or from the Sauce wiki, like you indicated? > > > > @mmerrell-sauce <https://github.com/mmerrell-sauce> agreed on the REST > > Assured stuff. I wasn't aware of another HTTP client, so I just went with > > this one. I will update to use some other HTTP client 👍 > > > > However, I would still like to clarify one point on which we seem to be > on > > a different page. Just for my understanding, are you suggesting that we > > should leave the integration of API and UI automation into a sample > > framework up to our client? > > > > This was actually the hardest part and nobody on the SA team knew how to > > combine API/UI automation into a single Cucumber framework that runs in > > parallel. As a result, I spent a few hours to resolve this problem. If we > > move this example into it is own repo or Git Gist, then we are leaving > the > > client with the responsibility of figuring out how to integrate API and > UI > > automation into a single framework and make it work correctly. > > > > If that's what we want to do as a team, then that's the decision. But it > > seems scary to me that a couple of expert SAs took hours to resolve this > > issue, meaning it was tough. And we will obscure that solution from that > > client and will just leave it on them to figure it out. > > > > Again, fine if we want to do that. I just wanted to make sure that we all > > understand the consequences of separating this example out 👍 > > > > — > > You are receiving this because your review was requested. > > Reply to this email directly, view it on GitHub > > < > #13 (comment) > >, > > or mute the thread > > < > https://github.com/notifications/unsubscribe-auth/AWZUEVbhEAW3z7vrBzrgBlzpDZRFJi7cks5vX6Y5gaJpZM4brxU1 > > > > . > > > > > -- > *Aaron Evans* > Test Automation Advisor | Sauce Labs > > 116 New Montgomery St #300 > San Francisco, CA 94105 > 855-677-0011 > > Sauce Labs <http://saucelabs.com/> | Sauce Blog <http://sauceio.com/> > > > > <https://saucelabs.com/> > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > < #13 (comment) >, > or mute the thread > < https://github.com/notifications/unsubscribe-auth/AmkEci-wLKRJTvXhtgDONXXvFkW_G-RYks5vX6oAgaJpZM4brxU1 > > . > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#13 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Arrvwmj2r_bHTfXlGhzKzGJxg2sHMsKxks5vX7A-gaJpZM4brxU1> .

-- *Marcus Merrell* Director of Technical Services | Sauce Labs 116 New Montgomery Street, Suite 300 San Francisco, CA 94105 (M) +1 512-619-9534 Sauce Labs <http://saucelabs.com/> | Sauce blog <http://saucelabs.com/blog>