Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(builder, server): Add Builds to Frontend and Packaged assets #7406

Closed
Closed
Show file tree
Hide file tree
Changes from 33 commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
48740d9
wip
aarushik93 Jul 11, 2024
7337e2f
most errors gone, i think
aarushik93 Jul 11, 2024
76cfeef
fixed input issue
aarushik93 Jul 11, 2024
5969d44
fix more issues
aarushik93 Jul 11, 2024
741e4db
number issue
aarushik93 Jul 11, 2024
2f4fe33
add static files
aarushik93 Jul 11, 2024
b376e84
feat: partial serving files on built version
ntindle Jul 12, 2024
bded791
feat: partial loading of routes and ui files
ntindle Jul 12, 2024
aedb8d8
fix: render all assets correctly
ntindle Jul 12, 2024
50d92d1
fix: move files to frontend
ntindle Jul 12, 2024
01cc25b
fix: import order
ntindle Jul 12, 2024
a57d27a
feat(dx): keep the frontend folder
ntindle Jul 12, 2024
42ee23d
fix: delete all the files in the frontend folder
ntindle Jul 12, 2024
d06d8a2
feat: update gitignore
ntindle Jul 12, 2024
fad8e30
Merge branch 'master' into aarushikansal/open-1370-serve-frontend-on-…
aarushik93 Jul 14, 2024
d3c8cf0
Merge branch 'master' into aarushikansal/open-1370-serve-frontend-on-…
ntindle Jul 15, 2024
2c56b85
ci(server): correctly build the server when builder changes
ntindle Jul 15, 2024
2e3e8ac
ci(server): correctly specify the npm install location
ntindle Jul 15, 2024
9fe1efa
fix(server): try a different way to specify working directory in a mo…
ntindle Jul 15, 2024
97581a7
fix(builder): typescript linting errors
ntindle Jul 15, 2024
2e6d094
Merge branch 'master' into aarushikansal/open-1370-serve-frontend-on-…
ntindle Jul 15, 2024
ab830b5
feat(server): more path protections and a warning
ntindle Jul 15, 2024
483cd07
Merge branch 'master' into aarushikansal/open-1370-serve-frontend-on-…
ntindle Jul 15, 2024
2a43462
Merge branch 'master' into aarushikansal/open-1370-serve-frontend-on-…
ntindle Jul 15, 2024
9be9dbe
Merge branch 'master' into aarushikansal/open-1370-serve-frontend-on-…
ntindle Jul 16, 2024
66384c1
fix(server): only mount the static files if the _next directory exists
ntindle Jul 16, 2024
75a5ee8
ci(server): prisma py fetch suggested after installing from windows
ntindle Jul 16, 2024
0716b59
ci(server): try this instead
ntindle Jul 16, 2024
8c212e5
ci(server): try and force prisma location
ntindle Jul 16, 2024
db14af2
ci(server): make the folder
ntindle Jul 16, 2024
be14969
fix(server): move the prisma fetch to build
ntindle Jul 16, 2024
5243c9d
Update setup.py
ntindle Jul 16, 2024
f673f8a
fix(server): comment
ntindle Jul 16, 2024
01e6fdd
ci(server): try more envars
ntindle Jul 16, 2024
20152cf
Update autogpt-server-ci.yml
ntindle Jul 16, 2024
5617a64
Update autogpt-server-ci.yml
ntindle Jul 16, 2024
a30b3f7
Merge branch 'master' into aarushikansal/open-1370-serve-frontend-on-…
ntindle Jul 16, 2024
94e2d0e
Update autogpt-server-ci.yml
ntindle Jul 16, 2024
c25f32e
Merge branch 'aarushikansal/open-1370-serve-frontend-on-server' of ht…
ntindle Jul 16, 2024
e46cfb1
Merge branch 'master' into aarushikansal/open-1370-serve-frontend-on-…
ntindle Jul 16, 2024
95ccc05
Merge branch 'master' into aarushikansal/open-1370-serve-frontend-on-…
ntindle Jul 16, 2024
6b89941
Update app.py
ntindle Jul 16, 2024
dc3a9b4
Merge branch 'aarushikansal/open-1370-serve-frontend-on-server' of ht…
ntindle Jul 16, 2024
e7c5696
Update rnd/autogpt_server/setup.py
ntindle Jul 16, 2024
a15ad5c
feat(server): breakpoints and cx_freeze update
ntindle Jul 17, 2024
2359a7d
Merge branch 'master' into aarushikansal/open-1370-serve-frontend-on-…
ntindle Jul 17, 2024
03df00c
feat(builder): merge continues
ntindle Jul 17, 2024
e676124
fix: spacing
ntindle Jul 17, 2024
72c6332
fix(builder): spacing
ntindle Jul 17, 2024
68f46c0
feat(builder, server): code review changes
ntindle Jul 17, 2024
bfb3a45
feat(server): try replace paths
ntindle Jul 17, 2024
7479d18
fix(server): remove debugpy
ntindle Jul 17, 2024
4565635
feat(server): try this instead
ntindle Jul 17, 2024
cbb81a3
fix(server): file name
ntindle Jul 17, 2024
189eccb
Update app.py
ntindle Jul 17, 2024
1ffa261
fix(server): setup remove replacement
ntindle Jul 17, 2024
f2849b4
Update rnd/autogpt_builder/src/components/CustomNode.tsx
aarushik93 Jul 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions .github/workflows/autogpt-server-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,13 @@ on:
paths:
- ".github/workflows/autogpt-server-ci.yml"
- "rnd/autogpt_server/**"
- "rnd/autogpt_builder/**"
pull_request:
branches: [master, development, release-*]
paths:
- ".github/workflows/autogpt-server-ci.yml"
- "rnd/autogpt_server/**"
- "rnd/autogpt_builder/**"

concurrency:
group: ${{ format('autogpt-server-ci-{0}', github.head_ref && format('{0}-{1}', github.event_name, github.event.pull_request.number) || github.sha) }}
Expand Down Expand Up @@ -144,6 +146,17 @@ jobs:
fetch-depth: 0
submodules: true

- uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '21'

- name: Install dependencies for autogpt_builder
working-directory: rnd/autogpt_builder
run: |
npm install
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yarn


- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
Expand Down Expand Up @@ -183,6 +196,13 @@ jobs:
- name: Install Python dependencies
run: poetry install

- name: Install Prisma Dependencies
run: |
mkdir prisma
poetry run prisma py fetch --force
env:
PRISMA_BINARY_CACHE_DIR: "./prisma"

- name: Generate Prisma Client
run: poetry run prisma generate

Expand Down
1 change: 1 addition & 0 deletions rnd/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
frontend/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • what's this for?
  • missing newline

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Staging area for cx-freeze to pull the built files from

9 changes: 8 additions & 1 deletion rnd/autogpt_builder/.eslintrc.json
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
{
"extends": "next/core-web-vitals"
"extends": [
"next/core-web-vitals",
"plugin:@typescript-eslint/recommended"
],
"rules": {
"@typescript-eslint/no-explicit-any": "off",
"@typescript-eslint/ban-ts-comment": "off"
}
}
11 changes: 10 additions & 1 deletion rnd/autogpt_builder/next.config.js
Original file line number Diff line number Diff line change
@@ -1,8 +1,17 @@
const dotenv = require('dotenv');
ntindle marked this conversation as resolved.
Show resolved Hide resolved
dotenv.config();

module.exports = {
const nextConfig = {
output: 'export',
ntindle marked this conversation as resolved.
Show resolved Hide resolved
env: {
AGPT_SERVER_URL: process.env.AGPT_SERVER_URL,
},
eslint: {
ignoreDuringBuilds: true,
},
typescript: {
ignoreBuildErrors: true,
},
};

module.exports = nextConfig
6 changes: 4 additions & 2 deletions rnd/autogpt_builder/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,9 @@
"class-variance-authority": "^0.7.0",
"clsx": "^2.1.1",
"date-fns": "^3.6.0",
"moment": "^2.30.1",
"dotenv": "^16.4.5",
"lucide-react": "^0.407.0",
"moment": "^2.30.1",
"next": "14.2.4",
"next-themes": "^0.3.0",
"react": "^18",
Expand All @@ -36,7 +36,9 @@
"@types/react": "^18",
"@types/react-dom": "^18",
"@types/react-modal": "^3.16.3",
"eslint": "^8",
"@typescript-eslint/eslint-plugin": "^7.16.0",
"@typescript-eslint/parser": "^7.16.0",
"eslint": "^8.57.0",
"eslint-config-next": "14.2.4",
"postcss": "^8",
"tailwindcss": "^3.4.1",
Expand Down
27 changes: 19 additions & 8 deletions rnd/autogpt_builder/src/app/build/page.tsx
Original file line number Diff line number Diff line change
@@ -1,8 +1,21 @@
"use client";
import React, { Suspense } from "react";
import Image from "next/image";
import { useSearchParams } from "next/navigation";
import FlowEditor from '@/components/Flow';

function FlowEditorWrapper() {
const searchParams = useSearchParams();
const flowID = searchParams.get("flowID") ?? undefined;

return (
<FlowEditor
className="flow-container w-full min-h-[75vh] border border-gray-300 dark:border-gray-700 rounded-lg"
flowID={flowID}
/>
);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unnecessary abstraction imo


export default function Home() {
return (
<div className="flex flex-col items-center px-12">
Expand Down Expand Up @@ -31,13 +44,11 @@ export default function Home() {
</a>
</div>
</div>

<div className="w-full flex justify-center mt-10">
<FlowEditor
className="flow-container w-full min-h-[75vh] border border-gray-300 dark:border-gray-700 rounded-lg"
flowID={useSearchParams().get("flowID") ?? undefined}
/>
</div>
<div className="w-full flex justify-center mt-10">
<Suspense fallback={<div>Loading...</div>}>
<FlowEditorWrapper />
</Suspense>
</div>
</div>
Comment on lines +47 to +52
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed indentation?

);
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing newline

2 changes: 1 addition & 1 deletion rnd/autogpt_builder/src/app/layout.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import { ThemeProvider as NextThemeProvider } from "next-themes";
import { type ThemeProviderProps } from "next-themes/dist/types";
import { Inter } from "next/font/google";
import Link from "next/link";
import { CubeIcon, Pencil1Icon, ReaderIcon, TimerIcon } from "@radix-ui/react-icons";
import { Pencil1Icon, TimerIcon } from "@radix-ui/react-icons";

import "./globals.css";

Expand Down
3 changes: 2 additions & 1 deletion rnd/autogpt_builder/src/app/monitor/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -266,6 +266,7 @@ const AgentFlowList = (
</Card>
);

// eslint-disable-next-line @typescript-eslint/no-unused-vars
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just put this in for now til i do a proper linting and formatting PR to get the build working

const FlowStatusBadge = ({ status }: { status: "active" | "disabled" | "failing" }) => (
<Badge
variant="default"
Expand Down Expand Up @@ -538,7 +539,7 @@ const FlowRunsTimeline = (
tickFormatter={s => s > 90 ? `${Math.round(s / 60)}m` : `${s}s`}
/>
<Tooltip
content={({ payload, label }) => {
content={({ payload}) => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

inconsistent spacing: { payload}

if (payload && payload.length) {
const data: FlowRun & { time: number, _duration: number } = payload[0].payload;
const flow = flows.find(f => f.id === data.flowID);
Expand Down
60 changes: 27 additions & 33 deletions rnd/autogpt_builder/src/components/CustomNode.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,16 @@ const CustomNode: FC<NodeProps<CustomNodeData>> = ({ data, id }) => {
setErrors((prevErrors) => ({ ...prevErrors, [key]: null }));
};

const getSelectValue = (value: any): string => {
if (typeof value === 'string') return value;
if (typeof value === 'number') return value.toString();
if (Array.isArray(value)) return value.join(',');
if (typeof value === 'object' && value !== null) {
return JSON.stringify(value);
}
return '';
};

const getValue = (key: string) => {
const keys = key.split('.');
return keys.reduce((acc, k) => (acc && acc[k] !== undefined) ? acc[k] : '', data.hardcodedValues);
Expand All @@ -121,7 +131,7 @@ const CustomNode: FC<NodeProps<CustomNodeData>> = ({ data, id }) => {
const isHandleConnected = (key: string) => {
return data.connections && data.connections.some((conn: any) => {
if (typeof conn === 'string') {
const [source, target] = conn.split(' -> ');
const [ target] = conn.split(' -> ');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

inconsistent spacing: [ target]

return target.includes(key) && target.includes(data.title);
}
return conn.target === id && conn.targetHandle === key;
Expand Down Expand Up @@ -168,11 +178,13 @@ const CustomNode: FC<NodeProps<CustomNodeData>> = ({ data, id }) => {
return <div className="connected-input">Connected</div>;
}

const renderClickableInput = (displayValue: string) => (
<div className="clickable-input" onClick={() => handleInputClick(fullKey)}>
{displayValue}
</div>
);
const renderClickableInput = (displayValue: unknown): JSX.Element => (
<div className="clickable-input" onClick={() => handleInputClick(fullKey)}>
{typeof displayValue === 'string' || typeof displayValue === 'number' || typeof displayValue === 'boolean'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{typeof displayValue === 'string' || typeof displayValue === 'number' || typeof displayValue === 'boolean'
{["string", "number", "boolean"].includes(typeof displayValue)

? String(displayValue)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for the three selected types, I think JSON.stringify does the same as String(v)/v.toString()

so then this whole thing can just be JSON.stringify(displayValue)

: JSON.stringify(displayValue)}
</div>
);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bad indentation


if (schema.type === 'object' && schema.properties) {
return (
Expand Down Expand Up @@ -297,7 +309,7 @@ const CustomNode: FC<NodeProps<CustomNodeData>> = ({ data, id }) => {
return schema.enum ? (
<div key={fullKey} className="input-container">
<select
value={value || ''}
value={getSelectValue(value)}
Comment on lines -303 to +313
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are there currently any cases where this is necessary?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's nothing in our nodes yet with different data types - but this fixes some of the TS check issues I was having - if there's a better solution lmk

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What was TypeScript's complaint?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Type error: Type '{ [key: string]: any; }' is not assignable to type 'string | number | readonly string[] | undefined'.

onChange={(e) => handleInputChange(fullKey, e.target.value)}
className="select-input"
>
Expand Down Expand Up @@ -337,7 +349,7 @@ const CustomNode: FC<NodeProps<CustomNodeData>> = ({ data, id }) => {
<div key={fullKey} className="input-container">
<input
type="number"
value={value || ''}
value={typeof value === 'number' ? value : typeof value === 'string' ? value : ''}
aarushik93 marked this conversation as resolved.
Show resolved Hide resolved
onChange={(e) => handleInputChange(fullKey, parseFloat(e.target.value))}
className="number-input"
/>
Expand All @@ -346,7 +358,7 @@ const CustomNode: FC<NodeProps<CustomNodeData>> = ({ data, id }) => {
);
case 'array':
if (schema.items && schema.items.type === 'string') {
const arrayValues = value || [];
const arrayValues = Array.isArray(value) ? value : [];
return (
<div key={fullKey} className="input-container">
{arrayValues.map((item: string, index: number) => (
Expand All @@ -357,7 +369,11 @@ const CustomNode: FC<NodeProps<CustomNodeData>> = ({ data, id }) => {
onChange={(e) => handleInputChange(`${fullKey}.${index}`, e.target.value)}
className="array-item-input"
/>
<Button onClick={() => handleInputChange(`${fullKey}.${index}`, '')} className="array-item-remove">
<Button onClick={() => {
const newArray = [...arrayValues];
newArray.splice(index, 1);
handleInputChange(fullKey, newArray);
}} className="array-item-remove">
&times;
</Button>
</div>
Expand All @@ -369,7 +385,7 @@ const CustomNode: FC<NodeProps<CustomNodeData>> = ({ data, id }) => {
</div>
);
}
return null;
return <div key={fullKey}></div>;
default:
return (
<div key={fullKey} className="input-container">
Expand All @@ -380,28 +396,6 @@ const CustomNode: FC<NodeProps<CustomNodeData>> = ({ data, id }) => {
}
};

const validateInputs = () => {
const newErrors: { [key: string]: string | null } = {};
const validateRecursive = (schema: any, parentKey: string = '') => {
Object.entries(schema.properties).forEach(([key, propSchema]: [string, any]) => {
const fullKey = parentKey ? `${parentKey}.${key}` : key;
const value = getValue(fullKey);

if (propSchema.type === 'object' && propSchema.properties) {
validateRecursive(propSchema, fullKey);
} else {
if (propSchema.required && !value) {
newErrors[fullKey] = `${fullKey} is required`;
}
}
});
};

validateRecursive(data.inputSchema);
setErrors(newErrors);
return Object.values(newErrors).every((error) => error === null);
};

return (
<div className={`custom-node dark-theme ${data.status === 'RUNNING' ? 'running' : data.status === 'COMPLETED' ? 'completed' : data.status === 'FAILED' ? 'failed' :''}`}>
<div className="node-header">
Expand Down
5 changes: 4 additions & 1 deletion rnd/autogpt_builder/src/components/Flow.tsx
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// @ts-nocheck
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bad

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just put this in for now til i do a proper linting and formatting PR to get the build working

"use client";
import React, { useState, useCallback, useEffect, useMemo } from 'react';
import ReactFlow, {
Expand Down Expand Up @@ -129,6 +130,7 @@ const FlowEditor: React.FC<{ flowID?: string; className?: string }> = ({
const onConnect: OnConnect = useCallback(
(connection: Connection) => {
setEdges((eds) => addEdge(connection, eds));

setNodes((nds) =>
nds.map((node) => {
if (node.id === connection.target) {
Expand Down Expand Up @@ -253,6 +255,7 @@ const FlowEditor: React.FC<{ flowID?: string; className?: string }> = ({
})));
}

// eslint-disable-next-line @typescript-eslint/no-unused-vars
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just put this in for now til i do a proper linting and formatting PR to get the build working

const prepareNodeInputData = (node: Node<CustomNodeData>, allNodes: Node<CustomNodeData>[], allEdges: Edge[]) => {
console.log("Preparing input data for node:", node.id, node.data.blockType);

Expand Down Expand Up @@ -285,7 +288,7 @@ const FlowEditor: React.FC<{ flowID?: string; className?: string }> = ({
return inputData;
};

let inputData = getNestedData(blockSchema, node.data.hardcodedValues);
const inputData = getNestedData(blockSchema, node.data.hardcodedValues);

console.log(`Final prepared input for ${node.data.blockType} (${node.id}):`, inputData);
return inputData;
Expand Down
4 changes: 2 additions & 2 deletions rnd/autogpt_builder/src/components/ui/calendar.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -60,8 +60,8 @@ function Calendar({
...classNames,
}}
components={{
IconLeft: ({ ...props }) => <ChevronLeftIcon className="h-4 w-4" />,
IconRight: ({ ...props }) => <ChevronRightIcon className="h-4 w-4" />,
IconLeft: () => <ChevronLeftIcon className="h-4 w-4" />,
IconRight: () => <ChevronRightIcon className="h-4 w-4" />,
}}
{...props}
/>
Expand Down
23 changes: 18 additions & 5 deletions rnd/autogpt_builder/tsconfig.json
Original file line number Diff line number Diff line change
@@ -1,13 +1,19 @@
{
"compilerOptions": {
"lib": ["dom", "dom.iterable", "esnext"],
"target": "esnext",
"module": "esnext",
"lib": [
"dom",
"dom.iterable",
"esnext"
],
"allowJs": true,
"skipLibCheck": true,
"strict": true,
"forceConsistentCasingInFileNames": true,
"noEmit": true,
"esModuleInterop": true,
"module": "esnext",
"moduleResolution": "bundler",
"moduleResolution": "node",
"resolveJsonModule": true,
"isolatedModules": true,
"jsx": "preserve",
Expand All @@ -21,6 +27,13 @@
"@/*": ["./src/*"]
}
},
"include": ["next-env.d.ts", "**/*.ts", "**/*.tsx", ".next/types/**/*.ts"],
"exclude": ["node_modules"]
"include": [
"next-env.d.ts",
"**/*.ts",
"**/*.tsx",
".next/types/**/*.ts"
],
"exclude": [
"node_modules"
]
}
Loading
Loading