diff --git a/CHANGELOG.md b/CHANGELOG.md
index a68e3303..4ed44e95 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -115,6 +115,9 @@
loaded (on CDP's `Network.loadingFinished`, not `Network.responseReceived`, so `exchange.response.body` is always
available), yielding the request's `Network::Exchange`. Doesn't require `network.intercept` to be set up, and is
never fired for requests that fail to load [#294]
+- `#evaluate`/`#execute` take named arguments that become the script's parameters, `page.evaluate("a + b", a: 1,
+ b: 2)`, a function declaration as the script, `await`, and `timeout:` [#673]
+- `#evaluate_handle` returning a `Ferrum::RemoteObject`, and `Node#execute`/`Node#evaluate_handle` [#673]
- `Ferrum::Browser#quit`/`Ferrum::Browser::Process#stop` accept `wait: false` to return immediately and run process
killing and user-data-directory cleanup on a background thread instead of blocking; the call returns the `Thread`
so callers can `#join` it if they need cleanup to have finished, e.g. before process exit or before reusing a
@@ -130,6 +133,10 @@
- `Ferrum::Page::Stream#stream` now closes the CDP stream handle (`IO.close`) once it's been fully read, so streams
opened for `Ferrum::Browser#pdf` and `Ferrum::Page::Tracing#record` no longer keep their backing storage alive in
the browser; a failed `IO.close` is raised to the caller.
+- `#evaluate_async`, `#evaluate_func` and `#evaluate_on` are deprecated, and scripts whose result will change in
+ 0.21 warn with a rewrite. 0.21 removes the old behavior, and `FERRUM_JS_API=new` runs the new one now, see
+ [upgrading](docs/13-javascript.md#upgrading-to-021) [#673]
+- `Ferrum::ScriptTimeoutError` is a `Ferrum::TimeoutError` [#673]
### Fixed
- An out-of-process iframe, e.g. Chrome's PDF viewer for an embedded PDF, appearing while a web worker was connected
diff --git a/README.md b/README.md
index 12358157..a0fe5fe6 100644
--- a/README.md
+++ b/README.md
@@ -85,6 +85,60 @@ JS
browser.quit
```
+Pass arguments by name, so the script can use parameters instead of digging
+through `arguments[0]`:
+
+```ruby
+page.evaluate("a + b", a: 1, b: 2) # => 3
+page.evaluate("el.getAttribute(name)", el: page.at_css("a"), name: "href")
+```
+
+A script that starts with a function or arrow declaration is used as-is, which
+is how you run more than one statement:
+
+```ruby
+page.evaluate(<<~JS, a: 1, b: 2)
+ function(a, b) {
+ const sum = a + b;
+ return sum * 3;
+ }
+JS
+```
+
+`await` works directly, and a function declaration's returned promise is
+awaited:
+
+```ruby
+page.evaluate("await fetch(url).then(r => r.text())", url: "/api")
+```
+
+Pass `timeout:` (seconds, defaulting to the page timeout) to bound how long the
+script may take before `Ferrum::ScriptTimeoutError` is raised, or `0` to let it
+run for as long as it takes. `timeout:` and `args:` are the only reserved
+keywords; use `args:` when a JavaScript parameter needs one of those names:
+
+```ruby
+page.evaluate("timeout * 2", args: { timeout: 21 }) # => 42
+```
+
+Use `#execute` when you only want the side effects, `#evaluate_handle` when you
+want to keep a value in the browser and pass it back in later, and
+`Ferrum::Node#evaluate` to run a script with `this` bound to an element:
+
+```ruby
+page.execute("window.scrollBy(0, 100)") # => true
+
+list = page.evaluate_handle("document.querySelectorAll('li')")
+page.evaluate("Array.from(nodes).map(n => n.textContent)", nodes: list)
+
+page.at_css("input").evaluate("this.value")
+```
+
+Scripts written for the old API keep working, and Ferrum warns, with a suggested
+rewrite, about any whose result will change. The next release, 0.21, removes the
+old behavior, so run your suite with `FERRUM_JS_API=new` to check it's ready. See
+[upgrading to 0.21](docs/13-javascript.md#upgrading-to-021).
+
Do any mouse movements you like:
```ruby
diff --git a/docs/13-javascript.md b/docs/13-javascript.md
index 1c8d028f..2f62f63e 100644
--- a/docs/13-javascript.md
+++ b/docs/13-javascript.md
@@ -4,16 +4,29 @@ sidebar_position: 13
# JavaScript
-#### evaluate(expression, \*args)
+#### evaluate(expression, \*args, timeout: nil, \*\*named)
-Evaluate and return result for given JS expression
+Evaluate JavaScript and return the result, converted to Ruby.
-* expression `String` should be valid JavaScript
-* args `Object` you can pass arguments, though it should be a valid `Node` or a
- simple value.
+* expression `String` a JavaScript expression, or a function/arrow declaration, which is used as-is and is how you
+ run more than one statement
+* args positional arguments, reachable as `arguments[0]` and up. Can't be mixed with named ones
+* timeout `Numeric` seconds to wait for the script and any promise it returns, defaults to the page timeout, `0`
+ waits for as long as it takes. Raises `Ferrum::ScriptTimeoutError`, a `Ferrum::TimeoutError`, when exceeded
+* named named arguments, which become the script's parameters. A `Ferrum::Node` arrives as the live element. Use
+ `args: { timeout: 1 }` for a parameter named `timeout` or `args`
```ruby
-page.evaluate("[window.scrollX, window.scrollY]")
+page.evaluate("[window.scrollX, window.scrollY]") # => [0, 0]
+page.evaluate("a + b", a: 1, b: 2) # => 3
+page.evaluate("el.getAttribute(name)", el: page.at_css("a"), name: "href")
+page.evaluate("await fetch(url).then(r => r.status)", url: "/") # => 200
+page.evaluate(<<~JS, c: 3) # => 9
+ function(c) {
+ const sum = 1 + 2;
+ return sum * c;
+ }
+JS
```
The result is converted to Ruby: objects become `Hash`, arrays and node lists `Array`, DOM nodes `Ferrum::Node`,
@@ -21,31 +34,58 @@ dates ISO 8601 strings, `NaN` and `Infinity` `Float`, `BigInt` `Integer`, `null`
without a Ruby counterpart, such as functions, `Map`, `Set` or `window`, become `{}`. A reference back to an
enclosing object, e.g. `a.self = a`, becomes `Ferrum::CyclicObject`.
-#### evaluate_async(expression, wait_time, \*args)
-
-Evaluate asynchronous expression and return result
+#### evaluate_handle(expression, \*args, timeout: nil, \*\*named) : `Ferrum::RemoteObject`
-* expression `String` should be valid JavaScript
-* wait_time How long we should wait for Promise to resolve or reject
-* args `Object` you can pass arguments, though it should be a valid `Node` or a
- simple value.
+Same as `evaluate`, but keeps the value in the browser and returns a reference you can pass back in as an argument.
+Call `#value` to convert it to Ruby, and `#release` to free it.
```ruby
-page.evaluate_async(%(arguments[0]({foo: "bar"})), 5) # => { "foo" => "bar" }
+list = page.evaluate_handle("document.querySelectorAll('li')")
+page.evaluate("Array.from(nodes).map(n => n.textContent)", nodes: list)
```
-#### execute(expression, \*args)
+#### execute(expression, \*args, timeout: nil, \*\*named) : `Boolean`
+
+Run JavaScript for its side effects. The script is used as a function body, so statements need no wrapping.
-Execute expression. Doesn't return the result
+```ruby
+page.execute("window.scrollBy(0, 100)") # => true
+```
-* expression `String` should be valid JavaScript
-* args `Object` you can pass arguments, though it should be a valid `Node` or a
- simple value.
+`Ferrum::Node` has the same three methods, with `this` bound to the element. Arrow functions don't get `this`, so use a
+`function` declaration when you need it.
```ruby
-page.execute(%(1 + 1)) # => true
+page.at_css("input").evaluate("this.value")
+page.at_css("input").execute("this.value = text", text: "hello")
```
+#### Upgrading to 0.21
+
+Scripts written for the old API run exactly as before, and Ferrum warns, pointing at your code, about any whose result
+will change. Calls that only the new API can make, e.g. with named arguments, already run the new way.
+
+* 0.20 runs old-style scripts the old way and warns. Set `FERRUM_JS_API=new` to run everything the new way.
+* 0.21 removes the old way, along with `evaluate_async`, `evaluate_func` and `evaluate_on`. There's no switch back.
+
+Warnings only cover the scripts and results your suite actually runs into, so before upgrading to 0.21, run the suite
+once with `FERRUM_JS_API=new`. Set `FERRUM_DEPRECATION_WARNINGS=raise` to turn the warnings into errors while
+migrating, or `0` to silence them. A library built on Ferrum should pass named arguments or a function declaration,
+which run the new way in both 0.20 and 0.21.
+
+| Warning | Old way | New way | Rewrite |
+|---|---|---|---|
+| Keyword arguments read through `arguments` | one object in `arguments[0]` | named parameters | `evaluate("options.foo", options: { foo: 1 })` |
+| Mixing positional and keyword arguments | keywords appended as an object | `ArgumentError` | pass the object in braces, or name every argument |
+| More than one expression, `"a(); b"` | returns `a()`, skips the rest | `SyntaxError` | `evaluate("function() { a(); return b }")` |
+| Never runs, a line break before the script | `nil`, nothing runs | runs and returns its value | check it's safe to run, or remove it |
+| Returns a promise | `{}`, not awaited | awaited, raises if rejected | prefix with `void` to keep it from waiting |
+| Returns an object with a `then` method | the object | awaited like a promise | wrap it, `({ value: object })` |
+| `Node#evaluate` returns a node, nodes or a `Date` | `{}` | `Ferrum::Node`, or the date's ISO 8601 string | check the code using the result |
+| `evaluate_async(expression, wait, *args)` | deprecated | removed | `evaluate("await …", timeout: wait)` |
+| `evaluate_func(function, *args, on:)` | deprecated | removed | `evaluate(function, a: 1)`, or `node.evaluate(function)` |
+| `evaluate_on(node:, expression:)` | deprecated | removed | `node.evaluate(expression)` |
+
#### evaluate_on_new_document(source, \*\*options) : `String`
Runs JavaScript in every new document of the page and its in-process frames, before the document's own scripts.
@@ -67,7 +107,7 @@ JS
#### isolated_world(name) : `IsolatedWorld`
Returns the frame's isolated world with this name, its own JavaScript globals over the frame's DOM, so the page's
-scripts can't see or override what runs there. It has `evaluate`, `evaluate_async`, `execute` and `evaluate_func`,
+scripts can't see or override what runs there. It has `evaluate`, `evaluate_handle` and `execute`,
takes and returns nodes, and comes back empty in every new document. On a page it's the main frame's world, use
`frame.isolated_world(name)` for an iframe. It's the same world `evaluate_on_new_document` runs in with
`world_name:`.
diff --git a/docs/21-workers.md b/docs/21-workers.md
index 7b8d3d10..32fc6e58 100644
--- a/docs/21-workers.md
+++ b/docs/21-workers.md
@@ -36,7 +36,7 @@ Returns service worker targets registered in this context. Unlike `workers`, the
are not connected to -- call `attach_target` first if you want to interact with one.
```ruby
-page.evaluate_async(%(navigator.serviceWorker.register("/sw.js").then(arguments[0])), 5)
+page.evaluate(%(await navigator.serviceWorker.register("/sw.js")))
sleep 0.1
@@ -91,7 +91,7 @@ Same API as `page.console` -- see [JavaScript](/docs/ferrum/javascript#console).
worker.console.map(&:text) # => ["from worker"]
```
-#### evaluate / execute / evaluate_async
+#### evaluate / execute / evaluate_handle
Same API as on `Page`/`Frame` -- see [JavaScript](/docs/ferrum/javascript). A worker has a single execution context,
so there's no frame to target.
diff --git a/lib/ferrum.rb b/lib/ferrum.rb
index bf1567af..db746feb 100644
--- a/lib/ferrum.rb
+++ b/lib/ferrum.rb
@@ -7,8 +7,10 @@
require "ferrum/utils/platform"
require "ferrum/utils/elapsed_time"
require "ferrum/utils/attempt"
+require "ferrum/utils/deprecate"
require "ferrum/errors"
require "ferrum/browser"
+require "ferrum/remote_object"
require "ferrum/node"
#
diff --git a/lib/ferrum/browser.rb b/lib/ferrum/browser.rb
index 6d698242..92378c43 100644
--- a/lib/ferrum/browser.rb
+++ b/lib/ferrum/browser.rb
@@ -36,7 +36,8 @@ class Browser
screenshot pdf mhtml viewport_size device_pixel_ratio
start_screencast stop_screencast
frames frame_by main_frame
- evaluate evaluate_on evaluate_async execute evaluate_func isolated_world
+ evaluate evaluate_handle execute
+ evaluate_on evaluate_async evaluate_func isolated_world
add_script_tag add_style_tag bypass_csp
evaluate_on_new_document remove_script_on_new_document
on position position=
diff --git a/lib/ferrum/client.rb b/lib/ferrum/client.rb
index 48af6f63..613f9a4a 100644
--- a/lib/ferrum/client.rb
+++ b/lib/ferrum/client.rb
@@ -209,7 +209,9 @@ def command(method, async: false, timeout: nil, **params)
#
# @param [Numeric, nil] timeout
# How long to wait for the browser to take the message, and then for its
- # response. Defaults to `protocol_timeout`.
+ # response. Defaults to `protocol_timeout`. `Float::INFINITY` waits for
+ # the response for as long as it takes, still giving the browser
+ # `protocol_timeout` to take the message.
#
# @return [Boolean, Hash]
# `true` when sent asynchronously, otherwise the parsed `"result"`
@@ -217,9 +219,10 @@ def command(method, async: false, timeout: nil, **params)
#
def send_message(message, async:, timeout: nil)
timeout ||= protocol_timeout
+ write_timeout = timeout.infinite? ? protocol_timeout : timeout
if async
- @ws.send_message(message, timeout)
+ @ws.send_message(message, write_timeout)
true
else
raise DeadBrowserError if @ws.messages.closed?
@@ -227,8 +230,8 @@ def send_message(message, async:, timeout: nil)
pending = Concurrent::IVar.new
@pendings[message[:id]] = pending
begin
- @ws.send_message(message, timeout)
- data = pending.value!(timeout)
+ @ws.send_message(message, write_timeout)
+ data = pending.value!(timeout.infinite? ? nil : timeout)
ensure
@pendings.delete(message[:id])
end
diff --git a/lib/ferrum/errors.rb b/lib/ferrum/errors.rb
index 298db890..57b95cec 100644
--- a/lib/ferrum/errors.rb
+++ b/lib/ferrum/errors.rb
@@ -87,7 +87,7 @@ def awaited
end
# Raised when an evaluated script takes too long to return a value.
- class ScriptTimeoutError < Error
+ class ScriptTimeoutError < TimeoutError
#
# Explains that the evaluated script timed out.
#
diff --git a/lib/ferrum/frame.rb b/lib/ferrum/frame.rb
index c53c344d..affa06a1 100644
--- a/lib/ferrum/frame.rb
+++ b/lib/ferrum/frame.rb
@@ -167,12 +167,11 @@ def parent
# frame.body # =>
lol
#
def content=(html)
- evaluate_async(%(
+ execute(<<~JS, html: html)
document.open();
- document.write(arguments[0]);
+ document.write(html);
document.close();
- arguments[1](true);
- ), @page.timeout, html)
+ JS
@page.document_node_id
end
alias set_content content=
diff --git a/lib/ferrum/frame/dom.rb b/lib/ferrum/frame/dom.rb
index 3cf1492b..44d34c05 100644
--- a/lib/ferrum/frame/dom.rb
+++ b/lib/ferrum/frame/dom.rb
@@ -27,32 +27,38 @@ class Frame
#
module DOM
SCRIPT_SRC_TAG = <<~JS
- const script = document.createElement("script");
- script.src = arguments[0];
- script.type = arguments[1];
- script.onload = arguments[2];
- document.head.appendChild(script);
+ function(url, type) {
+ const script = document.createElement("script");
+ script.src = url;
+ script.type = type;
+ document.head.appendChild(script);
+ return new Promise(resolve => script.onload = resolve);
+ }
JS
SCRIPT_TEXT_TAG = <<~JS
- const script = document.createElement("script");
- script.text = arguments[0];
- script.type = arguments[1];
- document.head.appendChild(script);
- arguments[2]();
+ function(content, type) {
+ const script = document.createElement("script");
+ script.text = content;
+ script.type = type;
+ document.head.appendChild(script);
+ }
JS
STYLE_TAG = <<~JS
- const style = document.createElement("style");
- style.type = "text/css";
- style.appendChild(document.createTextNode(arguments[0]));
- document.head.appendChild(style);
- arguments[1]();
+ function(content) {
+ const style = document.createElement("style");
+ style.type = "text/css";
+ style.appendChild(document.createTextNode(content));
+ document.head.appendChild(style);
+ }
JS
LINK_TAG = <<~JS
- const link = document.createElement("link");
- link.rel = "stylesheet";
- link.href = arguments[0];
- link.onload = arguments[1];
- document.head.appendChild(link);
+ function(url) {
+ const link = document.createElement("link");
+ link.rel = "stylesheet";
+ link.href = url;
+ document.head.appendChild(link);
+ return new Promise(resolve => link.onload = resolve);
+ }
JS
#
@@ -160,7 +166,7 @@ def xpath(selector, within: nil)
}
JS
- evaluate_func(expr, selector, within)
+ evaluate(expr, selector: selector, within: within)
end
#
@@ -187,7 +193,7 @@ def at_xpath(selector, within: nil)
return xpath.snapshotItem(0);
}
JS
- evaluate_func(expr, selector, within)
+ evaluate(expr, selector: selector, within: within)
end
#
@@ -214,7 +220,7 @@ def css(selector, within: nil)
}
JS
- evaluate_func(expr, selector, within)
+ evaluate(expr, selector: selector, within: within)
end
#
@@ -241,7 +247,7 @@ def at_css(selector, within: nil)
}
JS
- evaluate_func(expr, selector, within)
+ evaluate(expr, selector: selector, within: within)
end
#
@@ -307,17 +313,17 @@ def wait_for_selector(css: nil, xpath: nil, within: nil, timeout: @page.timeout,
# browser.add_script_tag(url: "http://example.com/stylesheet.css") # => true
#
def add_script_tag(url: nil, path: nil, content: nil, type: "text/javascript")
- expr, *args = if url
- [SCRIPT_SRC_TAG, url, type]
- elsif path || content
- if path
- content = File.read(path)
- content += "\n//# sourceURL=#{path}"
- end
- [SCRIPT_TEXT_TAG, content, type]
- end
+ if url
+ evaluate(SCRIPT_SRC_TAG, url: url, type: type)
+ elsif path || content
+ if path
+ content = File.read(path)
+ content += "\n//# sourceURL=#{path}"
+ end
+ evaluate(SCRIPT_TEXT_TAG, content: content, type: type)
+ end
- evaluate_async(expr, @page.timeout, *args)
+ true
end
#
@@ -333,17 +339,17 @@ def add_script_tag(url: nil, path: nil, content: nil, type: "text/javascript")
# browser.add_style_tag(content: "h1 { font-size: 40px; }") # => true
#
def add_style_tag(url: nil, path: nil, content: nil)
- expr, *args = if url
- [LINK_TAG, url]
- elsif path || content
- if path
- content = File.read(path)
- content += "\n//# sourceURL=#{path}"
- end
- [STYLE_TAG, content]
- end
+ if url
+ evaluate(LINK_TAG, url: url)
+ elsif path || content
+ if path
+ content = File.read(path)
+ content += "\n//# sourceURL=#{path}"
+ end
+ evaluate(STYLE_TAG, content: content)
+ end
- evaluate_async(expr, @page.timeout, *args)
+ true
end
end
end
diff --git a/lib/ferrum/frame/runtime.rb b/lib/ferrum/frame/runtime.rb
index b2cce4df..a8ff238b 100644
--- a/lib/ferrum/frame/runtime.rb
+++ b/lib/ferrum/frame/runtime.rb
@@ -1,6 +1,8 @@
# frozen_string_literal: true
require "singleton"
+require "ferrum/frame/script"
+require "ferrum/frame/runtime/legacy"
module Ferrum
#
@@ -21,19 +23,140 @@ def inspect
class Frame
#
- # Evaluates and executes JavaScript in a frame's execution context via
+ # Runs JavaScript in a frame's execution context via
# `Runtime.callFunctionOn`, converting arguments and return values
# between Ruby and JS, and resolving object/array/node results into Ruby
# equivalents. A reference back to an enclosing object becomes
# {CyclicObject}.
#
+ # There are three entry points, all of which accept the script in the
+ # same two shapes and pass arguments the same way:
+ #
+ # * {#evaluate} returns the value, serialized into Ruby.
+ # * {#evaluate_handle} returns a {RemoteObject} that stays in the browser.
+ # * {#execute} discards the value and returns `true`.
+ #
+ # ## Script shapes
+ #
+ # A script is either a bare expression, which gets wrapped in a function
+ # for you, or a function declaration, which is used as-is. Use the latter
+ # whenever you need more than one statement.
+ #
+ # page.evaluate("window.scrollY")
+ # page.evaluate("function() { const a = 1; return a + 1 }")
+ #
+ # Arrow functions keep the `this` of where they're defined, so on a
+ # {Node} use a `function` declaration when the script needs `this`.
+ #
+ # ## Arguments
+ #
+ # Keyword arguments become the function's parameters, so the script can
+ # name what it receives instead of digging through `arguments[0]`:
+ #
+ # page.evaluate("a + b", a: 1, b: 2)
+ #
+ # Each keyword is sent as its own protocol argument, so a {Node} arrives
+ # in JavaScript as the live element rather than as serialized JSON. When
+ # the script is a function declaration, arguments bind to its parameters
+ # by name, and one it doesn't declare raises an `ArgumentError`.
+ #
+ # Positional arguments still work and land in `arguments[0]` and up, but
+ # can't be mixed with keyword ones.
+ #
+ # `timeout:` and `args:` are reserved. Pass `args:` explicitly when you
+ # need a JavaScript parameter that happens to be named after one of them:
+ #
+ # page.evaluate("timeout * 2", args: { timeout: 21 })
+ #
+ # ## Promises
+ #
+ # A returned promise, or any object with a `then` method, is awaited, so
+ # `async`/`await` works directly and there is no separate asynchronous
+ # method:
+ #
+ # page.evaluate("await fetch(url).then(r => r.text())", url: "/api")
+ #
+ # ## Timeouts
+ #
+ # If the script hasn't settled within `timeout:` seconds (defaulting to
+ # the page's timeout) a {ScriptTimeoutError} is raised. `timeout: 0` lets
+ # it run for as long as it takes.
+ #
+ # The timeout is how long Ruby waits for Chrome's answer, so a page that
+ # replaces `Promise` or `setTimeout` can't defeat it, but it doesn't stop
+ # the script. A promise that never settles is harmless, while a
+ # synchronous endless loop keeps the page's JavaScript busy, so later
+ # calls on that page time out too. Other pages aren't affected.
+ #
+ # ## How a call runs
+ #
+ # All three methods, and their {Node} counterparts through
+ # {#evaluate_in}, take the same steps and differ only in a mode:
+ # `:value`, `:handle` or `:none`.
+ #
+ # 1. **Parse.** {Script} reads the source once, skipping strings,
+ # template literals, regular expressions and comments. It finds the
+ # code without surrounding comments and semicolons, whether it's a
+ # function or arrow declaration, its parameter names, and whether it
+ # uses `await`.
+ # 2. **Arguments.** `timeout:` and `args:` are taken out of the keywords.
+ # Positional arguments are passed as they are. Named ones bind to a
+ # declaration's parameters by name, or become the parameters of the
+ # function an expression is wrapped in.
+ # 3. **Wrap.** A declaration is sent as it is. An expression becomes
+ # `function() { return () }`, `async` when it uses
+ # `await`. {#execute} uses the script as written as the function body
+ # instead, so statements need no wrapping.
+ # 4. **Call.** One `Runtime.callFunctionOn` in the frame's context, or
+ # with `this` bound to a node or remote object, that awaits a returned
+ # promise. {Node} and {RemoteObject} arguments are passed by reference.
+ # 5. **Result.** {#evaluate} converts Chrome's deep serialization into
+ # Ruby, {#evaluate_handle} wraps an object in a {RemoteObject}, and
+ # {#execute} returns `true`. A thrown value raises {JavaScriptError}.
+ #
+ # For example, this call:
+ #
+ # page.evaluate("await fetch(url).then(r => r.status)", url: "/ping")
+ #
+ # is an expression that uses `await`, so it's sent as:
+ #
+ # async function(url) {
+ # return (
+ # await fetch(url).then(r => r.status)
+ # );
+ # }
+ #
+ # with the arguments `[{ value: "/ping" }]`. Chrome awaits the promise
+ # and answers `{ type: "number", value: 200 }`, which comes back as
+ # `200`. A declaration binds by name instead:
+ #
+ # node.evaluate("function(prefix, suffix) { return prefix + this.id + suffix }", suffix: "!", prefix: "#")
+ #
+ # is sent unchanged, with `this` bound to the node, the arguments
+ # reordered to `["#", "!"]`, and comes back as `"#main!"`.
+ #
+ # ## Scripts written for the old API
+ #
+ # Until Ferrum 0.21, a call that only the old API could have made runs
+ # the old way through {Legacy} and warns when its result will change.
+ # That's a call without named arguments, or one whose script reads
+ # `arguments` or also passes positional ones, unless the script is a
+ # declaration or uses `await`, or the method is {#evaluate_handle} or
+ # {Node#execute}.
+ # `FERRUM_JS_API=new` sends every call through the steps above.
+ #
module Runtime
+ include Legacy
+
INTERMITTENT_ATTEMPTS = ENV.fetch("FERRUM_INTERMITTENT_ATTEMPTS", 6).to_i
INTERMITTENT_SLEEP = ENV.fetch("FERRUM_INTERMITTENT_SLEEP", 0.1).to_f
DEEP_SERIALIZATION = {
serialization: "deep",
additionalParameters: { maxNodeDepth: 0, includeShadowTree: "none" }
}.freeze
+ # Describes a primitive or node result, which a handle can't stand for,
+ # without serializing an object that stays in the browser.
+ SHALLOW_SERIALIZATION = DEEP_SERIALIZATION.merge(maxDepth: 0).freeze
UNSERIALIZABLE_NUMBERS = {
"NaN" => Float::NAN,
"-0" => -0.0,
@@ -41,156 +164,283 @@ module Runtime
"-Infinity" => -Float::INFINITY
}.freeze
+ # Stands in for a declared parameter the caller didn't pass, which CDP
+ # sends as `undefined`.
+ UNDEFINED = Object.new.freeze
+ private_constant :UNDEFINED
+
#
- # Evaluate and return result for given JS expression.
+ # Evaluates JavaScript and returns the result, serialized into Ruby.
#
# @param [String] expression
- # The JavaScript to evaluate.
+ # A JavaScript expression, or a function declaration.
+ #
+ # @param [Array] positional
+ # Arguments reachable as `arguments[0]` and up. Can't be combined
+ # with named ones.
+ #
+ # @param [Hash] keywords
+ # Named arguments, which become the function's parameters, plus:
#
- # @param [Array] args
- # Additional arguments to pass to the JavaScript code.
+ # @option keywords [Numeric, nil] :timeout
+ # How long to wait for the script, and any promise it returns, to
+ # settle, in seconds. Defaults to the page timeout. `0` disables it.
+ #
+ # @option keywords [Hash] :args
+ # Named arguments whose names would collide with `timeout` or `args`.
+ #
+ # @return [Object]
+ # The result. DOM nodes come back as {Node}, arrays and plain objects
+ # are converted recursively, and cyclic values become {CyclicObject}.
+ #
+ # @raise [ScriptTimeoutError]
+ # The script didn't settle within `timeout`.
+ #
+ # @raise [JavaScriptError]
+ # The script threw.
#
# @example
- # browser.evaluate("[window.scrollX, window.scrollY]")
+ # browser.evaluate("[window.scrollX, window.scrollY]") # => [0, 0]
+ # browser.evaluate("a + b", a: 1, b: 2) # => 3
+ # browser.evaluate("await fetch(url).then(r => r.status)", url: "/") # => 200
#
- def evaluate(expression, *args)
- expression = format("function() { return %s }", expression)
- call(expression: expression, arguments: args)
+ def evaluate(expression, *positional, **keywords)
+ run(expression, positional, keywords, mode: :value)
end
#
- # Evaluate asynchronous expression and return result.
- #
- # @param [String] expression
- # The JavaScript to evaluate.
+ # Same as {#evaluate}, but returns a {RemoteObject} that keeps the value
+ # in the browser instead of serializing it. Handles can be passed back
+ # in as arguments. Primitives are returned as-is, and DOM nodes as
+ # {Node}, since both are already usable from Ruby.
#
- # @param [Integer] wait
- # How long we should wait for Promise to resolve or reject.
+ # @param (see #evaluate)
+ # @option (see #evaluate)
#
- # @param [Array] args
- # Additional arguments to pass to the JavaScript code.
+ # @return [RemoteObject, Node, Object]
#
# @example
- # browser.evaluate_async(%(arguments[0]({foo: "bar"})), 5) # => { "foo" => "bar" }
+ # list = page.evaluate_handle("document.querySelectorAll('li')")
+ # page.evaluate("Array.from(nodes).map(n => n.textContent)", nodes: list)
#
- def evaluate_async(expression, wait, *args)
- template = <<~JS
- function() {
- return new Promise((__f, __r) => {
- try {
- arguments[arguments.length] = r => __f(r);
- arguments.length = arguments.length + 1;
- setTimeout(() => __r(new Error("timed out promise")), %s);
- %s
- } catch(error) {
- __r(error);
- }
- });
- }
- JS
-
- expression = format(template, wait * 1000, expression)
- call(expression: expression, arguments: args, awaitPromise: true)
+ def evaluate_handle(expression, *positional, **keywords)
+ run(expression, positional, keywords, mode: :handle)
end
#
- # Execute expression. Doesn't return the result.
+ # Runs JavaScript for its side effects and discards the result. Unlike
+ # {#evaluate}, the script is used as a function body rather than an
+ # expression, so multiple statements need no wrapping.
#
- # @param [String] expression
- # The JavaScript to evaluate.
+ # @param (see #evaluate)
+ # @option (see #evaluate)
#
- # @param [Array] args
- # Additional arguments to pass to the JavaScript code.
+ # @return [Boolean]
+ # Always `true`.
#
# @example
- # browser.execute(%(1 + 1)) # => true
+ # browser.execute("window.scrollBy(0, 100)") # => true
+ # browser.execute(<<~JS, url: "/next")
+ # history.pushState({}, "", url);
+ # window.dispatchEvent(new Event("popstate"));
+ # JS
#
- def execute(expression, *args)
- expression = format("function() { %s }", expression)
- call(expression: expression, arguments: args, handle: false, returnByValue: true)
+ def execute(expression, *positional, **keywords)
+ run(expression, positional, keywords, mode: :none)
true
end
#
- # Evaluates a raw JS function declaration (unlike {#evaluate}, which
- # wraps the given expression in one), optionally on a specific remote
- # object instead of the frame's global execution context.
+ # @api private
#
- # @param [String] expression
- # A JS function declaration, e.g. `"function(a, b) { return a + b }"`.
+ # Backs {Node#evaluate}, {Node#evaluate_handle}, {Node#execute} and
+ # {RemoteObject#value}. Runs the script with `this` bound to `object`,
+ # using the same argument and script conventions as {#evaluate}.
#
- # @param [Array] args
- # Arguments to pass to the function.
+ # @param [Node, RemoteObject] object
+ # The node or object to bind `this` to.
#
- # @param [Node, nil] on
- # Remote object to invoke the function on.
+ # @param [Symbol] mode
+ # `:value`, `:handle` or `:none`.
#
- def evaluate_func(expression, *args, on: nil)
- call(expression: expression, arguments: args, on: on)
+ # @return [Object]
+ #
+ def evaluate_in(object, expression, positional, keywords, mode: :value)
+ run(expression, positional, keywords, mode: mode, on: object)
end
+ private
+
#
- # Evaluates an expression against a given node's remote object (+this+
- # refers to the node), returning the raw JS value rather than
- # resolving it to a {Node}/Hash/Array.
+ # Parses the script, turns the arguments into a parameter list and
+ # values, wraps the script into the function declaration that's sent,
+ # and hands it to {#call}. `mode` decides both how the result is
+ # converted and whether the script's value is kept.
#
- # @param [Node] node
- # The node to evaluate the expression on.
+ def run(expression, positional, keywords, mode:, on: nil)
+ script = Script.new(expression)
+ if legacy?(script, positional, keywords, mode: mode, on: on)
+ return run_legacy(script, positional, keywords, mode: mode, on: on)
+ end
+
+ named = keywords.except(:timeout, :args)
+ params, values = arguments_for(script, positional, named_arguments(keywords[:args], named), mode)
+ call(wrap(script, params, mode), values, on: on, mode: mode, timeout: script_timeout(keywords[:timeout]))
+ end
+
#
- # @param [String] expression
- # The JavaScript to evaluate.
+ # Normalizes the two argument styles into the parameter names of the
+ # wrapping function and the values to send:
#
- # @param [Boolean] by_value
- # Whether to return the plain JS value instead of a handle.
+ # * positional ones: no names, values reachable as `arguments[0]` and up;
+ # * named ones for a declaration: no names, values in the order of its
+ # own parameters;
+ # * named ones for an expression: their names, checked to be valid
+ # JavaScript identifiers, and their values.
#
- # @param [Integer] wait
- # Passed through to the underlying `Runtime.callFunctionOn` command.
+ def arguments_for(script, positional, named, mode)
+ if positional.any?
+ raise ArgumentError, "Pass arguments either positionally or by name, not both" if named.any?
+
+ return [[], positional]
+ end
+
+ return [[], []] if named.empty?
+ return [[], bind_to_parameters(script, named)] if function?(script, mode)
+
+ [named.keys.map { |key| validate_parameter!(key) }, named.values]
+ end
+
+ def named_arguments(args, named)
+ return named if args.nil?
+ raise ArgumentError, "args: takes a Hash of named arguments, got #{args.class}" unless args.is_a?(Hash)
+
+ args.merge(named)
+ end
+
#
- def evaluate_on(node:, expression:, by_value: true, wait: 0)
- options = { handle: true }
- expression = format("function() { return %s }", expression)
- options = { handle: false, returnByValue: true } if by_value
- call(expression: expression, on: node, wait: wait, **options)
+ # Orders named arguments to match a declaration's own parameter list, so
+ # that `evaluate("function(a, b) { … }", b: 2, a: 1)` binds by name.
+ # Parameters left out are passed as `undefined`. Falls back to insertion
+ # order when the parameters can't be read, e.g. when destructured.
+ #
+ def bind_to_parameters(script, named)
+ names = script.parameters
+ return named.values unless names
+
+ values = named.transform_keys(&:to_s)
+ unknown = values.keys - names
+ if unknown.any?
+ raise ArgumentError, "#{unknown.map(&:inspect).join(', ')} not declared by the function, " \
+ "which takes (#{names.join(', ')})"
+ end
+
+ last = names.rindex { |name| values.key?(name) }
+ names.take(last.to_i + 1).map { |name| values.fetch(name, UNDEFINED) }
end
- private
+ def validate_parameter!(key)
+ name = key.to_s
+ return name if name.match?(Script::JS_IDENTIFIER) && !Script::RESERVED_WORDS.include?(name)
- def call(expression:, arguments: [], on: nil, wait: 0, handle: true, **options)
+ raise ArgumentError, "#{name.inspect} is not a valid JavaScript parameter name"
+ end
+
+ # A named function in a function body is a statement defining it, so
+ # {#execute} runs it as a body rather than calling it.
+ def function?(script, mode)
+ script.declaration? && !(mode == :none && script.named_function?)
+ end
+
+ #
+ # Builds the function declaration that is actually sent to the browser:
+ # the caller's declaration as-is, or an expression wrapped in one.
+ #
+ # wrap(Script.new("a + b"), %w[a b], :value)
+ # # => "function(a, b) {\nreturn (\na + b\n);\n}"
+ # wrap(Script.new("await x()"), [], :none)
+ # # => "async function() {\nawait x()\n}"
+ #
+ def wrap(script, params, mode)
+ return script.code if function?(script, mode)
+
+ # A body keeps its trailing semicolons, which can be statements of
+ # their own, as in `while (busy());`. An expression is parenthesized on
+ # its own lines so that neither automatic semicolon insertion nor a
+ # trailing `//` comment in it can swallow the `return`.
+ body = if mode == :none then script.source
+ elsif script.code.empty? then ""
+ else "return (\n#{script.code}\n);"
+ end
+ "#{'async ' if script.awaits?}function(#{params.join(', ')}) {\n#{body}\n}"
+ end
+
+ # Waiting for the response is what bounds the script, so a page that
+ # replaces `Promise` or `setTimeout`, or throttles timers, can't defeat
+ # it.
+ def script_timeout(timeout)
+ seconds = (timeout || @page.timeout).to_f
+ seconds.positive? ? seconds : Float::INFINITY
+ end
+
+ #
+ # Sends the declaration with `Runtime.callFunctionOn`, retrying while
+ # the frame's context is being replaced unless `this` is bound to an
+ # existing node, and converts the result.
+ #
+ # @param [Symbol] mode
+ # `:value` resolves the result into Ruby, `:handle` returns a
+ # {RemoteObject}, and `:none` discards it.
+ #
+ def call(declaration, values, on:, mode:, timeout:)
# do not rescue -> retry if we operate on an existing node
errors = on ? [] : [NodeNotFoundError, NoExecutionContextError]
Utils::Attempt.with_retry(errors: errors, max: INTERMITTENT_ATTEMPTS, wait: INTERMITTENT_SLEEP) do
- params = options.dup
+ target = call_target(on)
+ target[:returnByValue] = true if mode == :none
+ target[:serializationOptions] = DEEP_SERIALIZATION if mode == :value
+ target[:serializationOptions] = SHALLOW_SERIALIZATION if mode == :handle
- params = params.merge(objectId: resolve_node(on)) if on
- params = params.merge(serializationOptions: DEEP_SERIALIZATION) if handle
+ response = call_function(declaration, values, target, timeout)
+ handle_error(response)
- if params[:executionContextId].nil? && params[:objectId].nil?
- params = params.merge(executionContextId: execution_id!)
+ result = response["result"]
+ case mode
+ when :none then nil
+ when :handle then handle_remote_object(result)
+ else handle_response(result)
end
+ end
+ end
- response = @page.command("Runtime.callFunctionOn",
- wait: wait, slowmoable: true,
- **params.merge(functionDeclaration: expression,
- arguments: prepare_args(arguments)))
- handle_error(response)
- response = response["result"]
+ def call_function(declaration, values, target, timeout)
+ @page.command("Runtime.callFunctionOn",
+ timeout: timeout, slowmoable: true, awaitPromise: true,
+ functionDeclaration: declaration, arguments: prepare_args(values), **target)
+ rescue TimeoutError
+ raise ScriptTimeoutError
+ end
- handle ? handle_response(response) : response["value"]
+ def call_target(on)
+ if on.is_a?(RemoteObject)
+ { objectId: on.remote_id }
+ elsif on
+ { objectId: resolve_node(on) }
+ else
+ { executionContextId: execution_id! }
end
end
- # FIXME: We should have a central place to handle all type of errors
def handle_error(response)
details = response["exceptionDetails"]
- return unless details
+ raise JavaScriptError, details if details
+ end
- case details.dig("exception", "description")
- when /\AError: timed out promise/
- raise ScriptTimeoutError
- else
- raise JavaScriptError, details
- end
+ def handle_remote_object(result)
+ return RemoteObject.new(self, @page, result) if result["objectId"] && result["subtype"] != "node"
+
+ handle_response(result)
end
def handle_response(response)
@@ -237,10 +487,15 @@ def resolve_node_options
def prepare_args(args)
args.map do |arg|
- if arg.is_a?(Node)
+ case arg
+ when Node
{ objectId: resolve_node(arg) }
- elsif arg.is_a?(Hash) && arg["objectId"]
- { objectId: arg["objectId"] }
+ when RemoteObject
+ { objectId: arg.remote_id }
+ when UNDEFINED
+ {}
+ when Hash
+ arg["objectId"] ? { objectId: arg["objectId"] } : { value: arg }
else
{ value: arg }
end
diff --git a/lib/ferrum/frame/runtime/legacy.rb b/lib/ferrum/frame/runtime/legacy.rb
new file mode 100644
index 00000000..1e1eb60f
--- /dev/null
+++ b/lib/ferrum/frame/runtime/legacy.rb
@@ -0,0 +1,359 @@
+# frozen_string_literal: true
+
+module Ferrum
+ class Frame
+ module Runtime
+ #
+ # The JavaScript evaluation API as it was before named arguments, kept
+ # so that scripts written for it return exactly what they used to. A
+ # call takes this path unless `FERRUM_JS_API=new` is set or the call
+ # uses something only the new API has: named arguments, a function
+ # declaration, `await`, or one of the new methods.
+ #
+ # Whatever runs here is checked for what Ferrum 0.21 will do
+ # differently, which is warned about with a suggested rewrite. 0.21
+ # removes this module, leaving only the new way.
+ #
+ # The script is parsed by {Script} as for the new way, to route the
+ # call and to spot what changes. An old-style call then runs as it did
+ # in Ferrum 0.19:
+ #
+ # 1. **Arguments.** Keyword arguments, `timeout:` and `args:` included,
+ # are appended to the positional ones as one trailing object.
+ # 2. **Wrap.** The source is wrapped as it is, in
+ # `function() { return }`, or `function() { }` for
+ # {#execute}. A line break after `return` ends that statement, and
+ # only the first statement of `"a(); b"` runs.
+ # 3. **Call.** One `Runtime.callFunctionOn` that doesn't await
+ # promises, bounded by the page timeout, which raises a plain
+ # {TimeoutError}. {#evaluate_async} races the promise against a
+ # timer in the page instead.
+ # 4. **Check.** Before the result is converted, it's compared with what
+ # 0.21 will do: its subtype for promises, nodes and dates, its
+ # properties for a `then` method or nodes in an array, and the
+ # parsed script for statements separated by a semicolon. Nothing is
+ # run on the result, so getters and proxy traps don't fire.
+ # 5. **Warn.** Each difference warns once per call site, naming the
+ # first caller outside installed gems, then the result is converted
+ # as before: a promise comes back as `{}`.
+ #
+ module Legacy
+ # Marker rejected browser-side by {#evaluate_async} when it times out.
+ TIMEOUT_MARKER = "timed out promise"
+
+ NEXT_VERSION = "Ferrum 0.21"
+
+ #
+ # @deprecated Will be removed in Ferrum 0.21. Use {#evaluate}, which awaits
+ # promises: `page.evaluate("await thing()", timeout: wait)`.
+ #
+ # Evaluates an asynchronous expression, appending a resolve callback as
+ # the last entry of `arguments`.
+ #
+ # @param [String] expression
+ # The JavaScript to evaluate.
+ #
+ # @param [Integer] wait
+ # How long to wait for the promise to settle, in seconds.
+ #
+ # @param [Array] args
+ # Additional arguments, reachable as `arguments[0]` and up. The
+ # resolve callback follows them.
+ #
+ # @return [Object]
+ #
+ def evaluate_async(expression, wait, *args)
+ Utils::Deprecate.warn(
+ "#{self.class}#evaluate_async will be removed in Ferrum 0.21. Use #evaluate, which waits for a " \
+ "returned promise, instead of calling the last of `arguments`: " \
+ "evaluate(\"await new Promise(resolve => setTimeout(resolve, 100))\", timeout: #{wait})."
+ )
+
+ template = <<~JS
+ function() {
+ return new Promise((__f, __r) => {
+ try {
+ arguments[arguments.length] = r => __f(r);
+ arguments.length = arguments.length + 1;
+ setTimeout(() => __r(new Error("#{TIMEOUT_MARKER}")), %s);
+ %s
+ } catch(error) {
+ __r(error);
+ }
+ });
+ }
+ JS
+
+ expression = format(template, wait * 1000, expression)
+ legacy_call(expression: expression, arguments: args, awaitPromise: true)
+ end
+
+ #
+ # @deprecated Will be removed in Ferrum 0.21. Use {#evaluate}, which takes a
+ # function declaration and named arguments, or {Node#evaluate}.
+ #
+ # Evaluates a raw JS function declaration, optionally on a specific
+ # remote object instead of the frame's global execution context.
+ #
+ # @param [String] expression
+ # A JS function declaration, e.g. `"function(a, b) { return a + b }"`.
+ #
+ # @param [Array] args
+ # Arguments to pass to the function.
+ #
+ # @param [Node, nil] on
+ # Remote object to invoke the function on.
+ #
+ # @return [Object]
+ #
+ def evaluate_func(expression, *args, on: nil)
+ Utils::Deprecate.warn(
+ "#{self.class}#evaluate_func will be removed in Ferrum 0.21. Use #evaluate, which takes a function " \
+ "declaration and named arguments: evaluate(\"function(a, b) { return a + b }\", a: 1, b: 2). " \
+ "For on:, use node.evaluate."
+ )
+
+ legacy_call(expression: expression, arguments: args, on: on)
+ end
+
+ #
+ # @deprecated Will be removed in Ferrum 0.21. Use {Node#evaluate}.
+ #
+ # Evaluates an expression against a given node's remote object (+this+
+ # refers to the node).
+ #
+ # @param [Node] node
+ # The node to evaluate the expression on.
+ #
+ # @param [String] expression
+ # The JavaScript to evaluate.
+ #
+ # @param [Boolean] by_value
+ # Whether to return the plain JS value instead of resolving it.
+ #
+ # @param [Integer] wait
+ # Passed through to the underlying `Runtime.callFunctionOn` command.
+ #
+ # @return [Object]
+ #
+ def evaluate_on(node:, expression:, by_value: true, wait: 0)
+ Utils::Deprecate.warn(
+ "#{self.class}#evaluate_on will be removed in Ferrum 0.21. Use node.evaluate(\"this.value\") instead."
+ )
+
+ options = { handle: true }
+ expression = format("function() { return %s }", expression)
+ options = { handle: false, returnByValue: true } if by_value
+ legacy_call(expression: expression, on: node, wait: wait, **options)
+ end
+
+ private
+
+ # Calls that only the new API could have made go the new way.
+ def legacy?(script, positional, keywords, mode:, on:)
+ return false if ENV.fetch("FERRUM_JS_API", nil) == "new"
+ return false if mode == :handle || (on && (mode == :none || !on.is_a?(Node)))
+ return false if function?(script, mode) || script.awaits?
+
+ keywords.empty? || positional.any? || script.uses?("arguments")
+ end
+
+ # Keywords used to be collected into a trailing Hash, `timeout:` and
+ # `args:` included, so they're passed on exactly that way.
+ def run_legacy(script, positional, keywords, mode:, on:)
+ warn_about_keywords(script, positional, keywords)
+ arguments = keywords.empty? ? positional : [*positional, keywords]
+
+ if mode == :none
+ legacy_execute(script, arguments)
+ elsif on
+ legacy_evaluate_on(script, arguments, on)
+ else
+ legacy_evaluate(script, arguments)
+ end
+ end
+
+ def legacy_evaluate(script, arguments)
+ warn_about_line_break(script)
+ response = legacy_call(expression: format("function() { return %s }", script.source),
+ arguments: arguments, handle: :raw, serializationOptions: DEEP_SERIALIZATION)
+ legacy_handle_error(response)
+ warn_about_result(script, response, :value)
+ warn_about_statements(script)
+ handle_response(response["result"])
+ end
+
+ def legacy_evaluate_on(script, arguments, node)
+ warn_about_line_break(script)
+ response = legacy_call(expression: format("function() { return %s }", script.source),
+ arguments: arguments, on: node, handle: :raw)
+ legacy_handle_error(response)
+ warn_about_result(script, response, :node)
+ warn_about_statements(script)
+ serialize(response["result"])
+ end
+
+ def legacy_execute(script, arguments)
+ response = legacy_call(expression: format("function() { %s }", script.source),
+ arguments: arguments, handle: :raw)
+ legacy_handle_error(response)
+ warn_about_result(script, response, :none)
+ release(response["result"])
+ true
+ end
+
+ # What `returnByValue` used to give, fetched afterwards so that the
+ # result could be inspected first.
+ def serialize(result)
+ object_id = result["objectId"]
+ return result["value"] unless object_id
+
+ response = @page.command("Runtime.callFunctionOn", objectId: object_id, returnByValue: true,
+ functionDeclaration: "function() { return this }")
+ response.dig("result", "value")
+ ensure
+ release(result)
+ end
+
+ def release(result)
+ object_id = result&.dig("objectId")
+ @page.command("Runtime.releaseObject", objectId: object_id) if object_id
+ rescue BrowserError
+ nil
+ end
+
+ def legacy_call(expression:, arguments: [], on: nil, wait: 0, handle: true, **options)
+ # do not rescue -> retry if we operate on an existing node
+ errors = on ? [] : [NodeNotFoundError, NoExecutionContextError]
+
+ Utils::Attempt.with_retry(errors: errors, max: INTERMITTENT_ATTEMPTS, wait: INTERMITTENT_SLEEP) do
+ params = options.dup
+
+ params = params.merge(objectId: resolve_node(on)) if on
+ params = params.merge(serializationOptions: DEEP_SERIALIZATION) if handle == true
+
+ if params[:executionContextId].nil? && params[:objectId].nil?
+ params = params.merge(executionContextId: execution_id!)
+ end
+
+ response = @page.command("Runtime.callFunctionOn",
+ wait: wait, slowmoable: true,
+ **params.merge(functionDeclaration: expression,
+ arguments: prepare_args(arguments)))
+ next response if handle == :raw
+
+ legacy_handle_error(response)
+ response = response["result"]
+
+ handle ? handle_response(response) : response["value"]
+ end
+ end
+
+ def legacy_handle_error(response)
+ details = response["exceptionDetails"]
+ return unless details
+
+ description = details.dig("exception", "description").to_s
+ raise ScriptTimeoutError if description.start_with?("Error: #{TIMEOUT_MARKER}")
+
+ raise JavaScriptError, details
+ end
+
+ def warn_about_keywords(script, positional, keywords)
+ return if keywords.empty?
+
+ if positional.any?
+ warn_about(script, "Mixing positional and keyword arguments will raise ArgumentError in #{NEXT_VERSION}, " \
+ "where keywords will become named parameters instead of a trailing object. Pass the " \
+ "object in braces, evaluate(\"…\", 1, { foo: 1 }), or name every argument.")
+ else
+ warn_about(script, "Keyword arguments will become named parameters in #{NEXT_VERSION} instead of " \
+ "arriving as one object in `arguments[0]`. Name the object instead: " \
+ "evaluate(\"options.foo\", options: { foo: 1 }).")
+ end
+ end
+
+ def warn_about_line_break(script)
+ return unless script.leading_line_break?
+
+ warn_about(script, "This script never runs: the line break before it ends the `return` it's placed " \
+ "after, so the result is always nil. #{NEXT_VERSION} will run it and return its value. " \
+ "Check it's safe to run, or remove it.")
+ end
+
+ def warn_about_statements(script)
+ return if script.leading_line_break? || !script.several_statements?
+
+ warn_about(script, "This script is more than one expression, which will be a SyntaxError in " \
+ "#{NEXT_VERSION}; today only the first statement runs. Use a function declaration " \
+ "and return the value: evaluate(\"function() { a(); return b }\").")
+ end
+
+ def warn_about_result(script, response, kind)
+ result = response["result"]
+ case result["subtype"]
+ when "promise" then warn_about_promise(script, kind)
+ when "node" then warn_about_node(script, "a DOM node", "Ferrum::Node") if kind == :node
+ when "date" then warn_about_node(script, "a Date", "its ISO 8601 string") if kind == :node
+ when "array" then warn_about_object(script, result, kind) if kind == :node
+ when nil, "proxy" then warn_about_object(script, result, kind) if result["type"] == "object"
+ end
+ end
+
+ def warn_about_promise(script, kind)
+ if kind == :none
+ warn_about(script, "This script returns a promise, which #execute will wait for in #{NEXT_VERSION}. " \
+ "Drop the `return`, or use `void`, to keep it from waiting.")
+ else
+ warn_about(script, "This script returns a promise, which comes back as {} today. #{NEXT_VERSION} " \
+ "will wait for it and return its value, or raise if it rejects. Prefix the " \
+ "expression with `void` to keep it from waiting.")
+ end
+ end
+
+ def warn_about_node(script, value, replacement)
+ warn_about(script, "Node#evaluate returns #{value} as {} today, and will return #{replacement} in " \
+ "#{NEXT_VERSION}. Check the code that uses the result.")
+ end
+
+ def warn_about_object(script, result, kind)
+ if thenable?(result)
+ warn_about(script, "This script returns an object with a `then` method, which #{NEXT_VERSION} " \
+ "will wait for like a promise. Wrap it, e.g. `({ value: object })`, to get the object.")
+ end
+ return unless kind == :node && result["subtype"] == "array"
+
+ nodes = properties(result, ownProperties: true).any? { |p| p.dig("value", "subtype") == "node" }
+ warn_about_node(script, "DOM nodes in an array", "Ferrum::Node") if nodes
+ end
+
+ def properties(result, **options)
+ @page.command("Runtime.getProperties", objectId: result["objectId"], **options)["result"]
+ end
+
+ def thenable?(result)
+ serialized = result["deepSerializedValue"]
+ if serialized
+ return serialized["type"] == "object" &&
+ serialized["value"].to_a.any? { |key, value| then_function?(key, value["type"]) }
+ end
+
+ properties(result, ownProperties: false, nonIndexedPropertiesOnly: true).any? do |property|
+ then_function?(property["name"], property.dig("value", "type"))
+ end
+ end
+
+ def then_function?(name, type)
+ name == "then" && type == "function"
+ end
+
+ def warn_about(script, message)
+ excerpt = script.code.lines.first.to_s.strip
+ excerpt = "#{excerpt[0, 60]}…" if excerpt.length > 60 || script.code.lines.size > 1
+ Utils::Deprecate.warn("#{message} Set FERRUM_JS_API=new to run it the #{NEXT_VERSION} way now." \
+ "\n script: #{excerpt}", user_code: true)
+ end
+ end
+ end
+ end
+end
diff --git a/lib/ferrum/frame/script.rb b/lib/ferrum/frame/script.rb
new file mode 100644
index 00000000..ce015e1a
--- /dev/null
+++ b/lib/ferrum/frame/script.rb
@@ -0,0 +1,267 @@
+# frozen_string_literal: true
+
+require "strscan"
+
+module Ferrum
+ class Frame
+ #
+ # @api private
+ #
+ # A script passed to {Runtime#evaluate} and friends. Reads just enough
+ # JavaScript to tell a function declaration from an expression, to find
+ # its parameters and where the code ends, skipping strings, template
+ # literals, regular expressions and comments so that a `//` inside
+ # `"http://…"` isn't mistaken for one.
+ #
+ # The source is scanned once, in {#initialize}, keeping a stack of open
+ # brackets and template literals. On the way it records where the code
+ # starts and ends, where the first top-level `(…)` and the commas inside
+ # it are, where the first top-level `{…}` closes, and whether `await`
+ # appears. Every question below is answered from those positions.
+ #
+ # @example
+ # script = Script.new("// sum\nasync function(a, b = 1) { return a + b };")
+ # script.code # => "async function(a, b = 1) { return a + b }"
+ # script.declaration? # => true
+ # script.parameters # => ["a", "b"]
+ # script.awaits? # => false
+ #
+ class Script
+ # A JavaScript identifier, Unicode letters included.
+ IDENTIFIER = /[\p{L}\p{Nl}_$][\p{L}\p{Nl}\p{Mn}\p{Mc}\p{Nd}\p{Pc}$]*/
+
+ FUNCTION_DECLARATION = /\A(?:async\s+)?function[\s*(]/
+ NAMED_FUNCTION = /\A(?:async\s+)?function\s*\*?\s*#{IDENTIFIER}/
+ SINGLE_PARAMETER_ARROW = /\A(?:async\s+)?(#{IDENTIFIER})\s*=>/
+ ASYNC_PREFIX = /\A(?:async\s*)?\z/
+ ARROW = /\A\s*=>/
+
+ # A parameter's name, followed by its default value if any.
+ PARAMETER = /\A(#{IDENTIFIER})\s*(?:=(?!>)|\z)/
+
+ JS_IDENTIFIER = /\A#{IDENTIFIER}\z/
+
+ # Words that can't name a parameter.
+ RESERVED_WORDS = %w[
+ arguments await break case catch class const continue debugger default delete do else enum eval export extends
+ false finally for function if implements import in instanceof interface let new null package private protected
+ public return static super switch this throw true try typeof var void while with yield
+ ].freeze
+
+ COMMENT = %r{//[^\n]*|/\*.*?(?:\*/|\z)}m
+ NOISE = /\s+|#{COMMENT}/
+ STRING = /"(?:\\.|[^"\\\n])*"?|'(?:\\.|[^'\\\n])*'?/m
+ TEMPLATE_CHUNK = /(?:\\.|[^`\\$]|\$(?!\{))*/m
+ REGEXP = %r{/(?:\\.|\[(?:\\.|[^\]\\\n])*\]|[^/\\\n\[])+/[A-Za-z]*}
+ WORD = /[\p{Alnum}_$]+/
+
+ # Characters ending a value, after which a `/` is a division.
+ VALUE_END = %r{[\p{Alnum}_$)\]\}"'`/]}
+
+ # Words after which a `/` starts a regular expression, not a division.
+ REGEXP_KEYWORDS = %w[return typeof instanceof in of new delete void throw case do else yield await].freeze
+
+ # @return [String] the script as given.
+ attr_reader :source
+
+ # @return [String] the script without surrounding whitespace, comments
+ # and trailing semicolons.
+ attr_reader :code
+
+ #
+ # @param [String] source
+ #
+ def initialize(source)
+ @source = utf8(source.to_s)
+ @code_start = @code_end = 0
+ @parameter_commas = []
+ @several_statements = false
+ @words = Set.new
+ @scanned = @source.valid_encoding?
+ @scanned ? scan : @code_end = @source.bytesize
+ @code = @source.byteslice(@code_start...@code_end)
+ end
+
+ #
+ # Whether the script is a complete function or arrow declaration,
+ # rather than an expression that happens to begin with one, such as the
+ # immediately invoked `function() { … }()`.
+ #
+ # @return [Boolean]
+ #
+ def declaration?
+ return false unless @scanned
+
+ arrow? || (@code.match?(FUNCTION_DECLARATION) && @body_end == @code_end)
+ end
+
+ #
+ # Whether the script is a declaration of a named function, which as a
+ # function body is a statement defining it rather than a value.
+ #
+ # @return [Boolean]
+ #
+ def named_function?
+ declaration? && @code.match?(NAMED_FUNCTION)
+ end
+
+ #
+ # Whether the script uses `await`, and so has to run in an `async`
+ # function.
+ #
+ # @return [Boolean]
+ #
+ def awaits?
+ uses?("await")
+ end
+
+ #
+ # Whether the script mentions the given word as code, rather than as a
+ # property name or inside a string or comment.
+ #
+ # @param [String] word
+ #
+ # @return [Boolean]
+ #
+ def uses?(word)
+ @words.include?(word)
+ end
+
+ #
+ # Whether a top-level semicolon separates the code into several
+ # statements, as in `"a(); b"`. Statements separated only by line
+ # breaks aren't detected, since telling `"a\nb"` from `"a\n.b()"` takes
+ # a parser.
+ #
+ # @return [Boolean]
+ #
+ def several_statements?
+ @several_statements
+ end
+
+ #
+ # Whether a line break comes before the code, which right after a
+ # `return` ends that statement.
+ #
+ # @return [Boolean]
+ #
+ def leading_line_break?
+ @scanned && !@code.empty? && @source.byteslice(0...@code_start).include?("\n")
+ end
+
+ #
+ # The parameter names of a declaration, or `nil` when they can't be
+ # read plainly, e.g. because of destructuring or rest parameters.
+ #
+ # @return [Array, nil]
+ #
+ def parameters
+ return unless @scanned
+
+ if (single = @code[SINGLE_PARAMETER_ARROW, 1])
+ return [single]
+ end
+ return unless @paren_close
+
+ bounds = [@paren_open, *@parameter_commas, @paren_close]
+ pieces = bounds.each_cons(2).map { |from, to| @source.byteslice((from + 1)...to).gsub(COMMENT, " ").strip }
+ pieces.pop if pieces.last.empty?
+ names = pieces.map { |piece| piece[PARAMETER, 1] }
+ names if names.all?
+ end
+
+ private
+
+ def utf8(source)
+ return source if source.encoding == Encoding::UTF_8
+ return source.dup.force_encoding(Encoding::UTF_8) if source.encoding == Encoding::BINARY
+
+ source.encode(Encoding::UTF_8)
+ rescue EncodingError
+ source.dup.force_encoding(Encoding::UTF_8)
+ end
+
+ def arrow?
+ return true if @code.match?(SINGLE_PARAMETER_ARROW)
+ return false unless @paren_close
+
+ @source.byteslice(@code_start...@paren_open).match?(ASYNC_PREFIX) &&
+ @source.byteslice((@paren_close + 1)..).match?(ARROW)
+ end
+
+ def scan
+ scanner = StringScanner.new(@source)
+ stack = []
+ previous = nil
+ separated = false
+
+ until scanner.eos?
+ next if scanner.skip(NOISE)
+
+ start = scanner.pos
+ previous = scan_token(scanner, stack, previous)
+ @code_start = start if @code_end.zero?
+ @code_end = scanner.pos unless previous == ";"
+ @several_statements ||= separated && previous != ";"
+ separated ||= previous == ";" && stack.empty?
+ end
+ end
+
+ def scan_token(scanner, stack, previous)
+ if scanner.scan(WORD)
+ @words << scanner.matched unless previous == "."
+ scanner.matched
+ elsif scanner.scan(STRING) || (regexp_allowed?(previous) && scanner.scan(REGEXP))
+ scanner.matched
+ elsif scanner.skip(/`/)
+ scan_template(scanner, stack)
+ else
+ scan_punctuator(scanner, stack)
+ end
+ end
+
+ def scan_punctuator(scanner, stack)
+ char = scanner.getch
+ track_parameters(char, scanner.pos - 1, stack)
+ case char
+ when "{", "(", "[" then stack.push(char)
+ when ")", "]" then stack.pop
+ when "}"
+ opened = stack.pop
+ return scan_template(scanner, stack) if opened == :template
+
+ @body_end ||= scanner.pos if opened == "{" && stack.empty?
+ end
+ char
+ end
+
+ # Remembers where the first top-level parenthesized group, the
+ # parameter list of a declaration, opens, closes and is split by commas.
+ def track_parameters(char, position, stack)
+ return if @paren_close
+
+ case char
+ when "(" then @paren_open ||= position if stack.empty?
+ when ")" then @paren_close = position if @paren_open && stack == ["("]
+ when "," then @parameter_commas << position if @paren_open && stack == ["("]
+ end
+ end
+
+ def scan_template(scanner, stack)
+ scanner.skip(TEMPLATE_CHUNK)
+ if scanner.skip(/\$\{/)
+ stack.push(:template)
+ else
+ scanner.skip(/`/)
+ end
+ "`"
+ end
+
+ def regexp_allowed?(previous)
+ return true if previous.nil? || REGEXP_KEYWORDS.include?(previous)
+
+ previous.length == 1 && !previous.match?(VALUE_END)
+ end
+ end
+ end
+end
diff --git a/lib/ferrum/node.rb b/lib/ferrum/node.rb
index 755d5561..9be126de 100644
--- a/lib/ferrum/node.rb
+++ b/lib/ferrum/node.rb
@@ -37,6 +37,7 @@ class Node
# The node's properties from Chrome's deep serialization.
#
def initialize(frame, properties)
+ @runtime = frame
@page = frame.page
@target_id = @page.target_id
@backend_node_id = properties["backendNodeId"]
@@ -242,9 +243,9 @@ def scroll_into_view
#
# @return [Boolean]
def in_viewport?(of: nil)
- function = <<~JS
- function(element, scope) {
- const rect = element.getBoundingClientRect();
+ evaluate(<<~JS, scope: of)
+ function(scope) {
+ const rect = this.getBoundingClientRect();
const [height, width] = scope
? [scope.offsetHeight, scope.offsetWidth]
: [window.innerHeight, window.innerWidth];
@@ -254,7 +255,6 @@ def in_viewport?(of: nil)
rect.right <= width;
}
JS
- page.evaluate_func(function, self, of)
end
#
@@ -401,15 +401,14 @@ def attribute(name)
# @return [Array]
#
def selected
- function = <<~JS
- function(element) {
- if (element.nodeName.toLowerCase() !== 'select') {
+ evaluate(<<~JS)
+ function() {
+ if (this.nodeName.toLowerCase() !== 'select') {
throw new Error('Element is not a