[Linux only] Upgrading to CEF 2704 #619

nethip · 2017-08-01T06:12:07Z

CEF verison upgrade on Linux to 2704.

Note: CEF version on Windows and Mac will still remain to be 2623. At a later time we will upgrade CEF on all platforms to one version.

The main reason why we could not update CEF on linux to 2171/2623 was because of app quit blockage. Upon App quit, the process was completely blocked as the app was waiting for one of the threads to join. After lot of debugging and analysis, I found out that this was because of Linux sandboxing, gtk_init() needs to be called after CefInitialize().

The other major change is the removal of libcrypto dependency, users now don't have to install this package for Brackets to run on Linux.

Here are other noteworthy changes.

From now on cefclient_gtk.cc will be the entry point for the app on Linux
I have modelled the enitre app to that of CefClient test app.
We are going to be using client::ClientHandler instead of appshell/ClientHandler. The difference is that the former will now derive from appshell/ClientHandler.
The App quit is now wired to RootWindowGtk close. client::RootWindowGtk is part of CefClient. All issues relating to app quit/ closing individual windows are now fixed.(Yay!!!).
Improved multiple windows workflow. g_handler is not going to be used anymore as a new client handler is created for every window creation by RootWindow itself. This improves our multiple windows workflow. Also Debug->New Window, will now have it's own menu bar (finally!). So each of the windows now have their own life time.
Menus creation and handling is now mapped to RootWindowGtk. RootWindowGtk will further deliver specific command to the respective browser.
appshell_extensions_gtk.cpp had an outdated version on master as it was always updated in the linux-1547 branch alone. So migrated all changes, relating to appshell_extensions_gtk.cpp present in linux-1547 branch to master.
All the changes are pertaining only to Linux. Wherever required I have wrapped all the code with #ifdef OS_LINUX macro
On Linux we are going to call startNodeProcess() at the start itself. If the thread, that spawns the node process, is called after CefInitialize(), it blocks the app quit.
Wired in the new developer tools.
Have modified cef_paths.gypi inside the redistributable for Linux 32 bit, zipped and uploaded that to S3.
Made changes to control file to remove the libcrypto dependency

Note to the committers: Whoever is merging, feel free to squash all the commits and merge.

Tasks

Install on Linux 14 and 16 LTS (both 32 and 64) systems and check if Brackets is running fine on these systems
Verify if libcrypto dependency is gone
Build this branch on all platforms and see Win and Mac are building fine.
Run unit tests
Sometimes, there is a performance bug with respect to startup, because of https://github.com/adobe/brackets/blob/master/src/filesystem/FileSystemEntry.js#L194. I could narrow down it down to recursion on startup. I will unit test more on different systems and take a call.

@ficristo, @ingorichter, @eyelash, @zaggino It would be great if you can review this PR.

…ed grunt updation

ficristo · 2017-08-01T09:35:44Z

Just to verify if I understood correctly: are you going to update macOS and Windows with cef 2704 ?
But in a different PR ?
Also, since you dind't use the current cef version used for macOS / Windows (2623) why you didn't use a newer version instead?
I'm not pressing to update it, I only want to know why.

nethip · 2017-08-01T10:41:37Z

@ficristo What I meant was that when we upgrade we will upgrade CEF on all the platforms to the latest version but for now only Brackets on Linux would be on 2704. Also I did not upgrade Win and Mac CEF versions to 2704 as I did not want Brackets to be effected on other platforms so kept the changes as minimal as possible and restricted only to Linux platform.

I tried updating to the latest CEF version, but was not successful building Linux atleast. The build mechanism has changed for building CefClient in the most recent versions of CEF(they moved away from gyp). So I guess we need to be update our build mechanism to that of the latest CefClient. We could do this later in a different PR.

swmitra · 2017-08-01T13:50:14Z

💯 👍 🥇

nethip · 2017-08-02T10:51:33Z

@ficristo Will you be able to review this PR and let me know if you have any comments?

ficristo · 2017-08-02T18:54:45Z

Sorry, I cannot really review C++ code.
Next week I should be able to, at least, try this PR. I just need to undust my VM...

@pratts you seemed interested in the past in Linux development of Brackets. Do you want to give a try to this PR ?

ficristo · 2017-08-05T08:48:48Z

In my light testing I can confirm the crash on quit is fixed.
It is also nice to have the developer tools inside Brackets itself.
I didn't try to build the installer.

I don't know how much it is important but when you scroll an element (the editor area, the file tree or the extension manager) I see the following line printed in the terminal:
[0805/104322:ERROR:PlatformKeyboardEvent.cpp(117)] Not implemented reached in static PlatformEvent::Modifiers blink::PlatformKeyboardEvent::getCurrentModifierState()

pratts · 2017-08-06T08:26:45Z

Thank you for this update @nethip. I really appreciate the effort you put in 💯 🥇 !
I cloned the brackets source code and checked out master for brackets and nethip/linux-cef-2704-upgrade for brackets-shell.
Brackets's grunt build and grunt installer worked fine. It no longer asked for libgcrypt11 👍
Trying to install the .deb file gives the following output:

prateek:/home/prateek/brackets$ sudo dpkg -i Brackets\ Release\ 1.11\ 64-bit.deb 
[sudo] password for prateek: 
Selecting previously unselected package brackets.
(Reading database ... 359397 files and directories currently installed.)
Preparing to unpack Brackets Release 1.11 64-bit.deb ...
Unpacking brackets (1.11.0-17496) ...
Setting up brackets (1.11.0-17496) ...
**xdg-icon-resource: size argument must be numeric
Try 'xdg-icon-resource --help' for more information.**
Processing triggers for menu (2.1.47) ...
Processing triggers for hicolor-icon-theme (0.13-1) ...

Running brackets for the first time from terminal gives the following error:
brackets
[0806/133905:FATAL:setuid_sandbox_host.cc(162)] The SUID sandbox helper binary was found, but is not configured correctly. Rather than run without sandboxing I'm aborting now. You need to make sure that /opt/brackets/chrome-sandbox is owned by root and has mode 4755.
#0 0x7fb81b590c3e base::debug::StackTrace::StackTrace()
#1 0x7fb81b5a59fb logging::LogMessage::~LogMessage()
#2 0x7fb81e5a3ad6 sandbox::SetuidSandboxHost::PrependWrapper()
#3 0x7fb81e4ee917 content::ZygoteCommunication::Init()
#4 0x7fb81e4ef31e content::CreateZygote()
#5 0x7fb81e218303 content::BrowserMainLoop::EarlyInitialization()
#6 0x7fb81e21ed79 content::BrowserMainRunnerImpl::Initialize()
#7 0x7fb81b502d19 CefMainDelegate::RunProcess()
#8 0x7fb81e6708e2 content::RunNamedProcessTypeMain()
#9 0x7fb81e671257 content::ContentMainRunnerImpl::Run()
#10 0x7fb81b496e6e CefContext::Initialize()
#11 0x7fb81b496c4b CefInitialize()
#12 0x7fb81b43acf6 cef_initialize
#13 0x0000005e1e38 CefInitialize()
#14 0x000000590c87 client::MainContextImpl::Initialize()
#15 0x000000570e7c client::(anonymous namespace)::RunMain()
#16 0x000000571571 main
#17 0x7fb817037b45 __libc_start_main
#18 0x00000055c779

Aborted

I changed the file permission using chmod 4755 /opt/brackets/chrome-sandbox

One more problem, I cannot see the menu bar :File,Edit,View, etc. Is it intentional ?

pratts · 2017-08-06T09:48:27Z

I can close the window using Ctrl+Q. It gives an error and window closes but I see an exception on console and brackets process does not terminate.

brackets
[0806/151520:ERROR:nss_util.cc(842)] After loading Root Certs, loaded==false: NSS error code: -8018
prateek@prateek-debian:~/work/open-source$ /opt/brackets/node-core/ConnectionManager.js:76
            console.error("[Connection] Unable to stringify message: " + e.message);
            ^

RangeError: Maximum call stack size exceeded
    at Connection._send (/opt/brackets/node-core/ConnectionManager.js:76:13)
    at Connection.sendEventMessage (/opt/brackets/node-core/ConnectionManager.js:203:14)
    at /opt/brackets/node-core/ConnectionManager.js:242:11
    at Array.forEach (native)
    at Object.sendEventToAllConnections (/opt/brackets/node-core/ConnectionManager.js:241:18)
    at Object.emitEvent (/opt/brackets/node-core/DomainManager.js:225:27)
    at EventEmitter.<anonymous> (/opt/brackets/node-core/BaseDomain.js:123:28)
    at emitThree (events.js:116:13)
    at EventEmitter.emit (events.js:194:7)
    at logReplacement (/opt/brackets/node-core/Logger.js:80:12)

nethip · 2017-08-07T05:08:28Z

Thanks @ficristo @pratts for trying this branch out and testing! Appreciate your efforts 👍

@ficristo I will dig deeper into it and see which module is reporting this error.

@pratts Thanks for your efforts. I will look into the issues you have mentioned above. I have done all the development on Ubuntu and there I have not seen this error. I will try to introduce the chmod through a grunt job. About the other console errors, I think the following is reported by Node side of things. I ran once into this error, but could not repro it again. I will try to repro this again and see.

RangeError: Maximum call stack size exceeded
    at Connection._send (/opt/brackets/node-core/ConnectionManager.js:76:13)
    at Connection.sendEventMessage (/opt/brackets/node-core/ConnectionManager.js:203:14)
    at /opt/brackets/node-core/ConnectionManager.js:242:11
    at Array.forEach (native)
    at Object.sendEventToAllConnections (/opt/brackets/node-core/ConnectionManager.js:241:18)
    at Object.emitEvent (/opt/brackets/node-core/DomainManager.js:225:27)
    at EventEmitter.<anonymous> (/opt/brackets/node-core/BaseDomain.js:123:28)
    at emitThree (events.js:116:13)
    at EventEmitter.emit (events.js:194:7)
    at logReplacement (/opt/brackets/node-core/Logger.js:80:12)

And about the Menus not being shown, are they currently shown if you build brackets-shell from linux-1547? Is this happening with this branch alone?

saurabh95 · 2017-08-07T05:01:24Z

appshell/appshell_extensions.cpp

+            #endif
+
+            #ifndef OS_LINUX
+                browser->GetHost()->ShowDevTools(wi, browser->GetHost()->GetClient(), settings, CefPoint());


This statement seems to be executed in case of MAC, so can we have a separate #elif case in line 406 only.
Something like #ifdef OS_MAC, it would make it more readable.

This statement needs to be executed for MAC as well.

I will see if I can simplify this for better readability.

saurabh95 · 2017-08-07T05:03:12Z

appshell/appshell_extensions.cpp

-		}
-		else {
+		} 
+        else if (message_name == "ReadDirWithStats") {


Could you please highlight the place where this message is called.

So here is the thing. In my testing what I figured out was readDir leads to lot of shell calls. Here is what happens for readDir. First we get plain contents of a directory and for every file entry, we then make a stat call to shell. With our Brackets repository open, this led to close to 33,000 shell calls. So if all we want is to get directory contents and file stats we could very well do that in a single shell call. Using this has reduced the no of shell calls to 5000 for readDir, for Brackets repo. So I guess this should pretty much be made mainstream. For now, I have just kept it added and in another PR, I would replace "ReadDir" shell call with "ReadDirWithStats". Does it make sense to remove this here and raise another PR for this?

saurabh95 · 2017-08-07T05:09:42Z

appshell/appshell_extensions_gtk.cpp

@@ -72,24 +93,9 @@
 #define KEY_BACK_QUOTE "`"
 #define KEY_COMMA ","

+typedef char s8;


This is not used anywhere. I guess we can remove it.

saurabh95 · 2017-08-07T05:11:03Z

appshell/common/client_app_delegates_common.cc

@@ -3,15 +3,16 @@
 // can be found in the LICENSE file.

 #include "appshell/common/client_app.h"
-#include "appshell/common/scheme_test_common.h"
+//#include "appshell/common/scheme_test_common.h"


saurabh95 · 2017-08-07T05:11:09Z

appshell/common/client_app_delegates_common.cc


 namespace client {

 // static
 void ClientApp::RegisterCustomSchemes(
    CefRefPtr<CefSchemeRegistrar> registrar,
    std::vector<CefString>& cookiable_schemes) {
-  scheme_test::RegisterCustomSchemes(registrar, cookiable_schemes);
+  // Brackets specific change.


saurabh95 · 2017-08-07T05:11:18Z

appshell/renderer/client_app_delegates_renderer.cc

@@ -4,14 +4,16 @@

 #include "appshell/renderer/client_app_renderer.h"
 #include "appshell/renderer/client_renderer.h"
-#include "appshell/renderer/performance_test.h"
+//#include "appshell/renderer/performance_test.h"


saurabh95 · 2017-08-07T05:11:25Z

appshell/renderer/client_app_delegates_renderer.cc


 namespace client {

 // static
 void ClientAppRenderer::CreateDelegates(DelegateSet& delegates) {
  renderer::CreateDelegates(delegates);
-  performance_test::CreateDelegates(delegates);
+  // Brackets specific change.


pratts · 2017-08-07T13:37:09Z

Hi @nethip
I can see the menu bar in build made from linux-1547 branch

I had to install libgcrypt11 library and saw the following error in terminal:

[0807/190453:ERROR:connection.cc(799)] sqlite error 11, errno 0: malformed database schema (is_transient) - near "where": syntax error
[0807/190453:ERROR:connection.cc(799)] sqlite error 11, errno 0: malformed database schema (is_transient) - near "where": syntax error
ATTENTION: default value of option force_s3tc_enable overridden by environment.
[0807/190514:ERROR:browser_main_loop.cc(212)] GLib-GObject: invalid cast from 'GtkExpandedContainer' to 'GtkWindow'
[0807/190514:ERROR:browser_main_loop.cc(212)] Gtk: IA__gtk_window_present_with_time: assertion 'GTK_IS_WINDOW (window)' failed

ingorichter · 2017-08-08T00:48:42Z

@nethip did you ask to test on Ubuntu? Which version of Ubuntu? 16/04, 16/10, 17/04? Older versions or other Linux distros?

nethip · 2017-09-06T06:02:16Z

I am in the process of creating a wiki for building brackets-shell on linux with the upgraded CEF and I will update the page with all possible errors that one might run into while building and the possible solutions for the same.

brackets 1.11, gcrypt.so file no longer needed, /usr included in source, remove seperate /usr packaging section fails to run, no errors, possible curl3 issue scala 2.12.3.1 plasma-integration added new dep, prep next build

nethip added 15 commits July 14, 2017 11:55

Initial changes to support CEF2623

f986980

Initial changes for 2623

1e00e5c

Intermediate changes

219a728

Prod level changes

8fc916e

CEFCLIENT_GTK cleanup

078c8e9

Mapped dev tools to newer client_handler

a2d697c

DispatchToBrowser() as well as startNodeProcess() changes

635a6d8

With quit working along with multiple windows quit workfloes

f080a82

Disabled controls for new popup windows

1e0a0c2

Menu Changes

9a0e979

Code cleanup

1d55890

LinuxVersionChanges

f9ba9ed

Incorporating CEF linux specific version changes along with ICU relat…

cb10755

…ed grunt updation

Adding installer dependencies

eba8b29

Code Cleanup

1b45720

Fixed windows build error

17b1f90

nethip added 2 commits August 1, 2017 23:31

Fixing issue with Linux 32 bit compilation

bd60656

Removing the libcrypt dependency from the installer script

df39ad7

saurabh95 reviewed Aug 7, 2017

View reviewed changes

jellemdekker mentioned this pull request Sep 25, 2017

fix libgcrypt dependency on debian 8 jessie #501

Closed

ficristo deleted the nethip/linux-cef-2704-upgrade branch September 26, 2017 18:26

This was referenced Sep 26, 2017

Brackets 1.8 does not support es6 in the UI portion of extensions in Linux adobe/brackets#13096

Closed

Linux version does not allow alternate source on startup adobe/brackets#6780

Open

Using x-tags with Brackets adobe/brackets#13405

Closed

sylhare mentioned this pull request Oct 30, 2017

How to install Brackets on Linux (Ubuntu) sylhare/webkit#1

Closed

petetnt mentioned this pull request Feb 18, 2018

Help for first pull request adobe/brackets#14121

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Linux only] Upgrading to CEF 2704 #619

[Linux only] Upgrading to CEF 2704 #619

nethip commented Aug 1, 2017 •

edited

Loading

ficristo commented Aug 1, 2017

nethip commented Aug 1, 2017 •

edited

Loading

swmitra commented Aug 1, 2017

nethip commented Aug 2, 2017

ficristo commented Aug 2, 2017

ficristo commented Aug 5, 2017

pratts commented Aug 6, 2017 •

edited

Loading

pratts commented Aug 6, 2017

nethip commented Aug 7, 2017

saurabh95 Aug 7, 2017

nethip Aug 8, 2017

nethip Aug 8, 2017

saurabh95 Aug 7, 2017

nethip Aug 8, 2017 •

edited

Loading

saurabh95 Aug 7, 2017

nethip Aug 8, 2017

saurabh95 Aug 7, 2017

saurabh95 Aug 7, 2017

saurabh95 Aug 7, 2017

saurabh95 Aug 7, 2017

pratts commented Aug 7, 2017

ingorichter commented Aug 8, 2017

nethip commented Sep 6, 2017 •

edited

Loading

[Linux only] Upgrading to CEF 2704 #619

[Linux only] Upgrading to CEF 2704 #619

Conversation

nethip commented Aug 1, 2017 • edited Loading

CEF verison upgrade on Linux to 2704.

ficristo commented Aug 1, 2017

nethip commented Aug 1, 2017 • edited Loading

swmitra commented Aug 1, 2017

nethip commented Aug 2, 2017

ficristo commented Aug 2, 2017

ficristo commented Aug 5, 2017

pratts commented Aug 6, 2017 • edited Loading

pratts commented Aug 6, 2017

nethip commented Aug 7, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nethip Aug 8, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pratts commented Aug 7, 2017

ingorichter commented Aug 8, 2017

nethip commented Sep 6, 2017 • edited Loading

nethip commented Aug 1, 2017 •

edited

Loading

nethip commented Aug 1, 2017 •

edited

Loading

pratts commented Aug 6, 2017 •

edited

Loading

nethip Aug 8, 2017 •

edited

Loading

nethip commented Sep 6, 2017 •

edited

Loading